From 3e9d810f93cfdecc87d4538e1721d08047d66010 Mon Sep 17 00:00:00 2001 From: henryzhongsc Date: Sat, 5 Oct 2024 09:10:28 -0500 Subject: [PATCH] =?UTF-8?q?=F0=9F=94=B0=20Update=20README=20to=20reflect?= =?UTF-8?q?=20Mamba=202=20support=20and=20its=20venv.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 03df85e..3629061 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,7 @@ We provide the minimum environment requirements to support the running of our pr * [`requirements/tf_4.40.txt`](https://github.com/henryzhongsc/longctx_bench/blob/main/requirements/tf_4.40.txt) supports the three transformers-based LLM baseline (`meta-llama/Meta-Llama-3-8B-Instruct`/`mistralai/Mistral-7B-Instruct-v0.2`/`lmsys/longchat-7b-v1.5-32k`), [Mamba](https://arxiv.org/abs/2312.00752), [Mamba-Chat](https://huggingface.co/havenhq/mamba-chat), [RecurrantGemma](https://storage.googleapis.com/deepmind-media/gemma/recurrentgemma-report.pdf), [RWKV-5-World](https://pypi.org/project/rwkv/), [FlexGen](https://arxiv.org/abs/2303.06865), [StreamingLLM](https://arxiv.org/abs/2309.17453), [InfLLM](https://arxiv.org/abs/2402.04617), [H2O](https://arxiv.org/abs/2306.14048), and [LLMLingua2](https://arxiv.org/abs/2403.12968). * [`requirements/tf_4.36.txt`](https://github.com/henryzhongsc/longctx_bench/blob/main/requirements/tf_4.36.txt) supports [KIVI](https://arxiv.org/abs/2402.02750) (it does also run LLM baselines, FlexGen, StreamingLLM, and InfLLM; but we opt to conduct such experiments in the above environment for maximum possible consistency). +* [`requirements/tf_4.45.txt`](https://github.com/henryzhongsc/longctx_bench/blob/main/requirements/tf_4.45.txt) supports [Mamba 2](https://arxiv.org/abs/2405.21060). We note that with `Transfofmers v4.42+` there is [a bugfix](https://github.com/huggingface/transformers/pull/30536) regarding KV cache duplication thanks to [@Cyrilvallez](https://www.linkedin.com/posts/cyril-vallez-070a53220_today-is-my-first-day-at-hugging-face-i-activity-7241439500936036352-_aYL?utm_source=share&utm_medium=member_desktop), so please be vigilant when comparing memory measurements across different environments. Should one be interested in reproducing a certain method, please look up the corresponding requirement file and install listed packages accordingly.