[WIP] Upstream encoder/decoder support based on multiple blocktables#3

Open

afeldman-nm wants to merge 60 commits intomainfrom aux_metadata3

+1,782-88

Commits on Feb 29, 2024

t5-small

Jin Shang
authored and
js8544
committed
fix
js8544
committed
lint
js8544
committed

Commits on Mar 1, 2024

Commits on Mar 6, 2024

Merge pull request #3 from afeldman-nm/enc_dec_t5
js8544
authored

Commits on Mar 7, 2024

arg naming fix
afeldman-nm
committed

Commits on Mar 12, 2024

Merge branch 'js8544_enc_dec_t5' into enc_dec_t5
afeldman-nm
committed
fixed attention_kernels.cu merge conflict; questions about ROCM
afeldman-nm
committed
llm_engine.py conflict resolution; removed prefix caching code; Sequence constructor call takes is_encoder_decoder, eos_token_id, lora_request calls; set is_encoder_decoder field in constructor
afeldman-nm
committed
actually updated Sequence constructor to take i_encoder_decoder, eos_token_id, lora_request arguments
afeldman-nm
committed
xformers.py accept incoming changes; replace paged_attention function with import of PagedAttentionImpl
afeldman-nm
committed
saved changed to xformers woops
afeldman-nm
committed
attempt at fixing model_runner conflicts related to encoder/decoder & prefix caching; low confidence of success
afeldman-nm
committed
encoder/decoder + prefix caching not supported; moved check from llm.py to model_runner.py
afeldman-nm
committed
refactoring, including: moved enc_dec_attention.py into vllm/model_executor/layers/attention
afeldman-nm
committed
existing regressions pass (yay) but encoder/decoder example fails
afeldman-nm
committed
fixed encoder/decoder reshape and cache bug, but paged attention calls are still incorrect
afeldman-nm
committed
augmented paged attention with context_lens, max_context_len, block_tables arguments to override input_metadata values; tests still pass but enc/dec still fails
afeldman-nm
committed
linting/formatting fixes
afeldman-nm
committed
Merge branch 'upstream-main' into enc_dec_t5_merge_upstream2
afeldman-nm
committed

Commits on Mar 19, 2024

Commits on Mar 22, 2024

Commits on Mar 26, 2024

first pass at T5 encoder support
afeldman-nm
committed

Commits on Mar 27, 2024

inefficient but effective & Attention-wrapper-compatible implementation of relative position encoding based on packed-variable-length-sequences
afeldman-nm
committed

Commits on Mar 28, 2024

wip cross-attention
afeldman-nm
committed

Commits on Apr 2, 2024

wip
afeldman-nm
committed