Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GPU] Force sdpa use onednn path for prefill and cl path for generation. #27387

Merged
merged 10 commits into from
Dec 4, 2024
Merged

[GPU] Force sdpa use onednn path for prefill and cl path for generation. #27387

merged 10 commits into from
Dec 4, 2024

Conversation

ceciliapeng2011
Copy link
Contributor

@ceciliapeng2011 ceciliapeng2011 commented Nov 4, 2024 •
edited by peterchen-intel
Loading

Details:

  • [GPU] Force SDPA use oneDNN path for prefill and clDNN path for generation on ARL-H platform
  • ...

Tickets:

@ceciliapeng2011 ceciliapeng2011 requested review from a team as code owners November 4, 2024 12:32
@github-actions github-actions bot added the category: GPU OpenVINO GPU plugin label Nov 4, 2024
@ceciliapeng2011 ceciliapeng2011 added do not merge and removed category: GPU OpenVINO GPU plugin labels Nov 4, 2024
@github-actions github-actions bot added the category: GPU OpenVINO GPU plugin label Nov 5, 2024
sshlyapn
sshlyapn approved these changes Nov 29, 2024
View reviewed changes
Copy link
Contributor

@sshlyapn sshlyapn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, left minor comments

src/plugins/intel_gpu/src/graph/impls/ocl/scaled_dot_product_attention.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/graph/impls/ocl/scaled_dot_product_attention.cpp Outdated Show resolved Hide resolved
src/plugins/intel_gpu/src/graph/impls/ocl/scaled_dot_product_attention.cpp Outdated Show resolved Hide resolved
@sshlyapn sshlyapn changed the title force sdpa use onednn path for prefill and cl path for generation. [GPU] Force sdpa use onednn path for prefill and cl path for generation. Nov 29, 2024
@p-durandin p-durandin added Code Freeze backport label to backport PR to the release branch labels Nov 29, 2024
@ceciliapeng2011 ceciliapeng2011 mentioned this pull request Dec 4, 2024
Merged
@yeonbok
Copy link
Contributor

yeonbok commented Dec 4, 2024

@ceciliapeng2011 LGTM, but is this only effective on ARLH? How about LNL?

@p-durandin p-durandin added this pull request to the merge queue Dec 4, 2024
github-merge-queue bot pushed a commit that referenced this pull request Dec 4, 2024
@ceciliapeng2011
[GPU] Force sdpa use onednn path for prefill and cl path for generati… (
82503ac 
#27889)

…on... the backport of pull/27387.

### Details:
- *[GPU] Force SDPA use oneDNN path for prefill and clDNN path for
generation on ARL-H platform*
- *backport of
[pull/27387](#27387

### Tickets:
 - *[CVS-158461](https://jira.devtools.intel.com/browse/CVS-158461)*
Merged via the queue into openvinotoolkit:master with commit 571e98d Dec 4, 2024
155 checks passed
@ceciliapeng2011
Copy link
Contributor Author

@ceciliapeng2011 LGTM, but is this only effective on ARLH? How about LNL?

Yes, it is only for ARL-H. We've also benchmarked with LNL and ARC, no strong justification for them... the performance differentiates among shape and models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@p-durandin p-durandin p-durandin approved these changes

@yeonbok yeonbok yeonbok approved these changes

@sshlyapn sshlyapn Awaiting requested review from sshlyapn

Assignees

@sshlyapn sshlyapn

Labels
backport label to backport PR to the release branch category: GPU OpenVINO GPU plugin Code Freeze
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ceciliapeng2011 @yeonbok @sshlyapn @p-durandin