#13204: adjust matmul program config selection for some sharded output scenarios #13819
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bbradelTT
requested review from
ayerofieiev-tt,
dmakoviichuk-tt,
rfurko-tt,
cfjchu,
TT-BrianLiu,
razorback3,
dongjin-na and
yugaoTT
as code owners
TT-BrianLiu
approved these changes
Oct 15, 2024
Comment on lines
+366
to
+375
| // MatmulMultiCoreProgramConfig does not support sharded output. | ||
| // Reduce in0_block_w if necessary to choose other configs. | ||
| if (mem_config.is_sharded() and Kt % in0_block_w != 0) { | ||
| in0_block_w = 1; | ||
| } | ||
|
|
||
| if (num_blocks_x * num_blocks_y <= num_cores_x * num_cores_y and Kt % in0_block_w == 0) { | ||
| CoreCoord core_range = get_core_range(num_blocks_y, num_blocks_x, num_cores_y, num_cores_x); | ||
| if (core_range.y == 1) { | ||
| bool use_mcast_config = mem_config.is_sharded() and core_range.y == 0; | ||
| if (core_range.y == 1 or (use_mcast_config and mem_config.memory_layout == TensorMemoryLayout::WIDTH_SHARDED)) { |
There was a problem hiding this comment.
Would it be better to do something like this for in0_block_w:
uint32_t in0_block_w = (Kt % 2 == 0) ? 2 : 1;
and remove the code from lines 366-371 and the check for Kt % in0_block_w == 0 on line 372?
There was a problem hiding this comment.
Yes it would. Unfortunately, that would require updating the PCCs for models/tests. I'm hoping to refactor this code, and will look at it then.
bbradelTT
force-pushed
the
bbradel-13204_out
branch
from
4cc5332 to
fe55daf
Compare
ct-clmsn
pushed a commit
to ct-clmsn/tt-metal
that referenced
this pull request
Nov 12, 2024
…arded output scenarios (tenstorrent#13819) * tenstorrent#13204: adjust matmul program config selection for some sharded output scenarios * tenstorrent#13204: adjust matmul program config selection for some height/block sharded output scenarios * tenstorrent#13204: add interleaved input sharded output matmul test
azecevicTT
added a commit
to tenstorrent/tt-mlir
that referenced
this pull request
Nov 13, 2024
Workaround introduced in #894 is not needed anymore. The issue was fixed in metal tenstorrent/tt-metal#13819. Closes #891 FYI @odjuricicTT
azecevicTT
added a commit
to tenstorrent/tt-mlir
that referenced
this pull request
Nov 13, 2024
Workaround introduced in #894 is not needed anymore. The issue was fixed in metal tenstorrent/tt-metal#13819. Closes #891 FYI @odjuricicTT
azecevicTT
added a commit
to tenstorrent/tt-mlir
that referenced
this pull request
Nov 14, 2024
Workaround introduced in #894 is not needed anymore. The issue was fixed in metal tenstorrent/tt-metal#13819. Closes #891 FYI @odjuricicTT
azecevicTT
added a commit
to tenstorrent/tt-mlir
that referenced
this pull request
Nov 14, 2024
Workaround introduced in #894 is not needed anymore. The issue was fixed in metal tenstorrent/tt-metal#13819. Closes #891 FYI @odjuricicTT
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ticket
Link to Github Issue #13204
Problem description
The matmul program config selection for a single core where Kt is not a multiple of 2 and output is sharded leads to choosing a program config that cannot handle sharded outputs
What's changed
Add checks for if the output is sharded, and choose an appropriate in0_block_w and mcast config
Checklist
T3k frequent tests passes https://github.com/tenstorrent/tt-metal/actions/runs/11351278842