New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Example of using key_padding_mask for flash attention v2 #530

Open

YJYJLee opened this issue Sep 8, 2023 · 1 comment

YJYJLee commented Sep 8, 2023

Could anyone provide a good example of using key_padding_mask for flash attention v2?
key_padding_mask is used to mask out the keys (shape [B, 1, 1, N])
Is it possible to use key_padding_mask and causal mask together?

Contributor

sz128 commented Sep 12, 2023

https://github.com/lm-sys/FastChat/blob/main/fastchat/train/llama_flash_attn_monkey_patch.py

samvanstroud mentioned this issue

[v2] Attention Masking #352

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment