-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GPU] Fix issue for skipping gather #21887
[GPU] Fix issue for skipping gather #21887
Conversation
465a430
to
aee3c36
Compare
Also as I mentioned in the offline chat, I think we can move do_rintime... stuffs to be done after empty primitive skipping. Because if the node is empty we do not need to do anything for that op already. Currently, the issue seems to happen for empty input primitive + its user's do_runtime_... check where it is trying to do something with its input mem. However we just can skip that user node if it is empty, before do the mem check and update things for those user nodes. So the priority of the optimization should be like this: runtime empty primitive skipping => further optimization in runtime (where we check the parent's memory, etc) |
c61dec9
to
34429b6
Compare
Note: To add corresponding test as a follow up work |
34429b6
to
c4b60a3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
c4b60a3
to
c786de1
Compare
- checkout input layout is zero count before calling dep_memory() - add the case for input layout count is zero - in gather_inst, run build_deps before checking is_the_same_buffer(output_memory(), input_memory())
…ntime_skip_gather do_runtime_in_place_kv_cache
… current node is skpped for empty output tensor
…empty and indices is empty
…front of chekcing skip execution
Details:
Tickets: