Code

Add t0 scripts#50

Code

Closed

Muennighoff wants to merge 14 commits intobigscience-workshop:masterbigscience-workshop/bigscience:masterfrom Muennighoff:t0scriptsMuennighoff/bigscience:t0scriptsCopy head branch name to clipboard

+607

Commits

Commits on Jul 4, 2022

• Add t0 scripts

  Muennighoff

  committed
  94adbdd

• Add T0 specific args

  Muennighoff

  committed
  3b8cc31

• Remove warmup

  Muennighoff

  committed
  058e60b

• Update train/t0/tr11f-6B3-ml-t0.slurm

  Show description for d93e169
  

  Muennighoff

  and

  thomasw21

  authored
  d93e169

• t0 -> tr13

  Muennighoff

  committed
  78791e2

• t0 -> tr13

  Muennighoff

  committed
  094a273

Commits on Jul 5, 2022

• Use weighted-split-path

  Muennighoff

  committed
  2fc43b0

Commits on Jul 9, 2022

• Add 350M script & adjust HPs

  Muennighoff

  committed
  41c1ef4

• Adjust tr13f-350M

  Muennighoff

  committed
  877158b

Commits on Jul 10, 2022

• Add weight decay based on FLAN

  Muennighoff

  committed
  5674c69

Commits on Jul 11, 2022

• Remove finetune & add checkpoint-activations

  Muennighoff

  committed
  582ee32

• Remove finetune

  Muennighoff

  committed
  d5b1622

Commits on Jul 14, 2022

• Change BS for more throughput & increase LR

  Muennighoff

  committed
  56729d3

Commits on Jul 16, 2022

• Update train/tr13-t0/t0_test.slurm

  Show description for 318dfc7
  

  Muennighoff

  and

  thomasw21

  authored
  318dfc7