Skip to content

Commit

Permalink
🔰 Update README to reflect Mamba 2 support and its venv.
Browse files Browse the repository at this point in the history
  • Loading branch information
henryzhongsc committed Oct 5, 2024
1 parent ace2663 commit 3e9d810
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ We provide the minimum environment requirements to support the running of our pr

* [`requirements/tf_4.40.txt`](https://github.com/henryzhongsc/longctx_bench/blob/main/requirements/tf_4.40.txt) supports the three transformers-based LLM baseline (`meta-llama/Meta-Llama-3-8B-Instruct`/`mistralai/Mistral-7B-Instruct-v0.2`/`lmsys/longchat-7b-v1.5-32k`), [Mamba](https://arxiv.org/abs/2312.00752), [Mamba-Chat](https://huggingface.co/havenhq/mamba-chat), [RecurrantGemma](https://storage.googleapis.com/deepmind-media/gemma/recurrentgemma-report.pdf), [RWKV-5-World](https://pypi.org/project/rwkv/), [FlexGen](https://arxiv.org/abs/2303.06865), [StreamingLLM](https://arxiv.org/abs/2309.17453), [InfLLM](https://arxiv.org/abs/2402.04617), [H2O](https://arxiv.org/abs/2306.14048), and [LLMLingua2](https://arxiv.org/abs/2403.12968).
* [`requirements/tf_4.36.txt`](https://github.com/henryzhongsc/longctx_bench/blob/main/requirements/tf_4.36.txt) supports [KIVI](https://arxiv.org/abs/2402.02750) (it does also run LLM baselines, FlexGen, StreamingLLM, and InfLLM; but we opt to conduct such experiments in the above environment for maximum possible consistency).
* [`requirements/tf_4.45.txt`](https://github.com/henryzhongsc/longctx_bench/blob/main/requirements/tf_4.45.txt) supports [Mamba 2](https://arxiv.org/abs/2405.21060). We note that with `Transfofmers v4.42+` there is [a bugfix](https://github.com/huggingface/transformers/pull/30536) regarding KV cache duplication thanks to [@Cyrilvallez](https://www.linkedin.com/posts/cyril-vallez-070a53220_today-is-my-first-day-at-hugging-face-i-activity-7241439500936036352-_aYL?utm_source=share&utm_medium=member_desktop), so please be vigilant when comparing memory measurements across different environments.

Should one be interested in reproducing a certain method, please look up the corresponding requirement file and install listed packages accordingly.

Expand Down

0 comments on commit 3e9d810

Please sign in to comment.