Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a wrong reference to seqlen_k variable in the varlen kernel #1192

Closed
wants to merge 2 commits into from
Closed

Conversation

cakeng
Copy link

@cakeng cakeng commented Aug 30, 2024

This fixes a line in compute_attn_1rowblock_splitkv kernel function that uses params.seqlen_k to use binfo.actual_seqlen_k instead to access the seqlen_k value.

This is a fix to the flash_attn_varlen_func function producing wrong results when used with cudaGraph over different input parameters, as found in the vLLM repo (vllm-project/vllm#6052).

@cakeng cakeng closed this by deleting the head repository Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant