Code

PagedAttention operation#867

Code

Closed

ilya-lavrenov wants to merge 22 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino_contrib:masterfrom ilya-lavrenov:paged-attentionilya-lavrenov/openvino_contrib:paged-attentionCopy head branch name to clipboard

+1,170-3

Commits

Commits on Feb 2, 2024

• Introduced first version of PagedAttention operation

  ilya-lavrenov

  committed
  dc4553a

• Added CPU implementation for reshape_and_cache

  ilya-lavrenov

  committed
  1542b7a

• Attention can be compiled

  ilya-lavrenov

  committed
  cbeff36

• Temp commit

  ilya-lavrenov

  committed
  cb708a0

Commits on Feb 5, 2024

• Creare SPDA operation for prefill case

  ilya-lavrenov

  committed
  b340cb3

• Finalize PagedAttention implementation

  ilya-lavrenov

  committed
  b6c35d6

• Added framework map to map to PyTorch module

  ilya-lavrenov

  committed
  e9d5e44

• Use Tensor::set_shape instead of view

  ilya-lavrenov

  committed
  04e7014

Commits on Feb 7, 2024

• Intermediate state

  ilya-lavrenov

  committed
  675de7b

Commits on Feb 11, 2024

• Passes accuracy

  ilya-lavrenov

  committed
  980fabc

Commits on Feb 12, 2024

• Merge remote-tracking branch 'upstream/master' into paged-attention

  ilya-lavrenov

  committed
  b36e57a

Commits on Feb 13, 2024

• No explicit registration of frontend::OpExtension, removed unnecessary extension opset.

  slyalin

  committed
  7f9ae41

Commits on Feb 14, 2024

• Merge pull request #1 from slyalin/simplified_extension_registration

  Show description for e1eda1a
  ilya-lavrenov

  authored
  e1eda1a

Commits on Feb 15, 2024

• Merge remote-tracking branch 'upstream/master' into paged-attention

  ilya-lavrenov

  committed
  3a368f8

• Remove print of performance

  ilya-lavrenov

  committed
  5b17873

• Supported GQA, MQA

  ilya-lavrenov

  committed
  f4bc7c8

Commits on Feb 17, 2024

• microsoft/phi-1_5 works

  ilya-lavrenov

  committed
  264e86d

Commits on Feb 18, 2024

• Suppport mistral with sliding_windows

  ilya-lavrenov

  committed
  793350d

• Fixed sliding_window

  ilya-lavrenov

  committed
  c4705ac

Commits on Feb 19, 2024

• Passing alibi_slopes and sliding_window to PagedAttension kernel

  slyalin

  committed
  bf920e3

• Fixed overflow with maximal sliding_window

  slyalin

  committed
  4e1cfd3

• Merge pull request #2 from slyalin/paged_attention_remaining_parameters

  Show description for d6388a9
  ilya-lavrenov

  authored
  d6388a9