Code

Fix glu activation#148

Code

Merged

thomasw21 merged 6 commits intomainbigscience-workshop/Megatron-DeepSpeed:mainfrom thomas/fix_glu_activationbigscience-workshop/Megatron-DeepSpeed:thomas/fix_glu_activationCopy head branch name to clipboardOct 22, 2021

+21-5

Commits

Commits on Oct 21, 2021

• Make sure to use glu activation when specified

  thomasw21

  committed
  e184e4b

  Copy full SHA for e184e4b

• Woops forgot DS config

  thomasw21

  committed
  da33290

  Copy full SHA for da33290

• Upsample ffn_hidden_size when glu is used

  thomasw21

  committed
  c83529f

  Copy full SHA for c83529f

• Woops

  thomasw21

  committed
  a45d4a2

  Copy full SHA for a45d4a2

• Replace assert with raising exception instead

  thomasw21

  committed
  57deccb

  Copy full SHA for 57deccb

Commits on Oct 22, 2021

• fix bug

  stas00

  committed
  f9dd27d

  Copy full SHA for f9dd27d