Code

Quantized: Use cublas for prompt#238

Code

Merged

EricLBuehler merged 15 commits intoEricLBuehler:masterEricLBuehler/mistral.rs:masterfrom lucasavila00:cublas_promptlucasavila00/mistral.rs:cublas_promptCopy head branch name to clipboardMay 15, 2024

+73-21

Commits

Commits on Apr 28, 2024

• Use cublas for prompt

  lucasavila00

  committed
  58fe2bc

• cublas prompt

  lucasavila00

  committed
  62d5206

Commits on Apr 29, 2024

• use f16 for output mul

  lucasavila00

  committed
  0347130

• mulmat via f16

  lucasavila00

  committed
  e069667

• disable attn mask in bench

  lucasavila00

  committed
  97c0324

• reduce contiguous calls

  lucasavila00

  committed
  699e541

• remove attn mask disabling

  lucasavila00

  committed
  6388053

• reduced precision, refactor

  lucasavila00

  committed
  0ca7255

• Merge branch 'master' into cublas_prompt

  lucasavila00

  committed
  a0d5f92

• cliipy

  lucasavila00

  committed
  4419381

Commits on May 14, 2024

• merge master

  lucasavila00

  committed
  f906dbc

• changes

  lucasavila00

  committed
  9d65c2d

• Update mistralrs-core/src/models/quantized_llama.rs

  EricLBuehler

  authored
  43ee0ad

• Update mistralrs-core/src/models/quantized_llama.rs

  EricLBuehler

  authored
  64f656b

Commits on May 15, 2024

• Merge branch 'master' into cublas_prompt

  EricLBuehler

  authored
  0979c5f