-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update eltwise flatbuffer to include extra params, move eltwise composite to own files #1121
Conversation
|
||
namespace tt::runtime::ttnn::operations::unary::composite { | ||
|
||
static void runEltwiseUnaryCompositeOP( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do we mean when we say "Composite" op in our runtime?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It reflects the scheme of ttnn (in src/tt-metal/ttnn/cpp/ttnn/operations/eltwise/unary/unary_composite.hpp
). They have a concept of unaryCompositeOp that has a different execution function signature than ordinary unary ops (same for binary_composite). Alot of these ops have extra parameters aside from the input tensor like min/max for clamp.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see ok makes sense.
DEBUG_ASSERT((*rhs)->is_allocated()); | ||
|
||
// Switch the order of operands if the second operand requires broadcast | ||
if ((*rhs)->volume() < (*lhs)->volume()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should guard this underneath a workaround.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, FYI @uazizTT I think you added this, is this still required in runtime? Are there plans to do this in the compiler?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created issue #1124
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, FYI @uazizTT I think you added this, is this still required in runtime? Are there plans to do this in the compiler?
We don't intend to fix it in the compiler as it should be fixed in tt-metal, this is the reported issue in tt-metal to track it. tenstorrent/tt-metal#13566
Once tt-metal fixes it on their end, this workaround can be removed, but until then it can be guarded as a workaround in runtime.
278a774
to
0af336b
Compare
… composite to its own file to match ttnn implementation
0af336b
to
0b5043d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Related #1093
As we're bringing up more ops, having a single flatbuffer table that describes parameters for eltwise ops isn't sufficient to cover all cases. For example, unary composite ops like clamp could have 2 extra float parameters min and max. These ops are however still modelled as eltwise operations in ttnn, so we should model them as eltwise in runtime as well.
This PR introduces a new union type
EltwiseOpParams
, which is a collection of extra parameters required for specific eltwise ops. These extra params can later be extracted in runtime to input into the ttnn op. This will allow us to add eltwise ops that have extra parameters in the future.For example, if we were to add a ClampOp as an EltwiseOp, we could create special params for it:
Then handle it in
TTNNToFlatbuffer.cpp
with something like:Then in runtime extract the required params.
This PR also moves unary/binary composite ops to their own files to match the implementation of ttnn and to reduce the size of unary.cpp and binary.cpp (especially as we add more composite ops).
FYI @mmanzoorTT