From 440d9ecbe53d8edeaafcf1b44edb6d2fd540f0ed Mon Sep 17 00:00:00 2001 From: Simon Willison Date: Thu, 24 Oct 2024 22:20:43 -0700 Subject: [PATCH] I may be wrong about this --- python/installing-flash-attention.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/installing-flash-attention.md b/python/installing-flash-attention.md index 82cc15ca7..514dd9e6a 100644 --- a/python/installing-flash-attention.md +++ b/python/installing-flash-attention.md @@ -6,9 +6,9 @@ pip install flash-attn --no-build-isolation ``` **Do not try to do this**. It is a trap. For some reason attempting to install this runs a compilation process which can take _multiple hours_. I tried to run this in Google Colab on an A100 machine that I was paying for and burned through $2 worth of "compute units" and an hour and a half of waiting before I gave up. -Thankfully [I learned](https://twitter.com/Sampson4242/status/1849666226299281443) that there's an alternative: the Flash Attention team provide pre-built wheels for their project exclusively through GitHub releases. They're not uploaded to PyPI, possibly because they're 180MB each or maybe because PyPI can't automatically pick the correct torch version? +> **Update**: I may be wrong about this, [the setup.py](https://github.com/Dao-AILab/flash-attention/blob/c1d146cbd5becd9e33634b1310c2d27a49c7e862/setup.py#L54-L56) for the project includes code that attempts to install wheels directly from the GitHub releases. That didn't work for me and I don't understand why. -Whatever the reason, you can find them attached to the most recent release on https://github.com/Dao-AILab/flash-attention/releases +Thankfully [I learned](https://twitter.com/Sampson4242/status/1849666226299281443) that there's an alternative: the Flash Attention team provide pre-built wheels for their project exclusively through GitHub releases. You can find them attached to the most recent release on https://github.com/Dao-AILab/flash-attention/releases But which one should you use out of the 83 files listed there?