Code

ggml : add ggml_soft_max_ext#4256

Code

Merged

ggerganov merged 14 commits intomasterggerganov/llama.cpp:masterfrom gg/soft-max-extggerganov/llama.cpp:gg/soft-max-extCopy head branch name to clipboardDec 1, 2023

+298-183

Commits

Commits on Nov 29, 2023

• metal : implement soft_max_ext

  ggerganov

  committed
  e89597c

  Copy full SHA for e89597c

• cuda : implement soft_max_ext

  ggerganov

  committed
  88519fb

  Copy full SHA for 88519fb

• ggml : implement soft_max_ext (CPU)

  ggerganov

  committed
  6a66f69

  Copy full SHA for 6a66f69

• batched-bench : print threads

  Show description for 390a445
  ggerganov

  committed
  390a445

  Copy full SHA for 390a445

• metal : simplify soft_max encoding

  Show description for 580fe20
  ggerganov

  committed
  580fe20

  Copy full SHA for 580fe20

Commits on Nov 30, 2023

• cuda : use 512 threads for soft_max instead of 32

  ggerganov

  committed
  ebd062b

  Copy full SHA for ebd062b

• ggml : update soft max cpu

  ggerganov

  committed
  c7c8dab

  Copy full SHA for c7c8dab

• cuda : do warp-based block reduce

  ggerganov

  committed
  62532c0

  Copy full SHA for 62532c0

• cuda : increase max block size to 1024

  ggerganov

  committed
  6b86bcf

  Copy full SHA for 6b86bcf

• cuda : fix warp reduction initialization of shared mem

  ggerganov

  committed
  68e02c0

  Copy full SHA for 68e02c0

• metal : warp-based reduction for soft max kernel

  ggerganov

  committed
  55717c9

  Copy full SHA for 55717c9

• metal : warp-based reduce for rms_norm

  ggerganov

  committed
  c4db592

  Copy full SHA for c4db592

Commits on Dec 1, 2023

• metal : simplify soft max kernel

  Show description for d9c8fa3
  ggerganov

  committed
  d9c8fa3

  Copy full SHA for d9c8fa3

• alloc : fix build with debug

  ggerganov

  committed
  eb594c0

  Copy full SHA for eb594c0