TTNN throws an exception if second operand of binary Ops require broadcast

[uazizTT]

TTNN throws the following exception when trying to broadcast second operand of a binary op:

RuntimeError: TT_THROW @ tt-metal/src/tt-metal/ttnn/cpp/ttnn/operations/eltwise/binary/device/binary_device_operation.cpp:46: tt::exception

TT_THROW("ttnn::operations::binary::BinaryDeviceOperation: unsupported broadcast");

Apply canonicalization on the operands or allow broadcasting for all operands.

[eyonland]

Please attach a pytest to reproduce this exact case.

Also, note that the workaround we currently have for broadcasting is to use the ttnn.bcast. For example https://github.com/tenstorrent/tt-metal/blob/main/models/demos/t3000/falcon40b/tt/model_utils.py#L504

[umadevimcw]

@uazizTT Kindly provide the examples to debug further on this issue

[uazizTT]

@umadevimcw Just trying to understand what kind of example you need, if I send you a test case from tt-mlir compiler that asserts at this bug, does that work for you?

[KalaivaniMCW]

@uazizTT Yes any unit test to reproduce would be fine.
Could you specify the binary op which you tested and the shapes and dtypes and memory_config used.
At present broadcast support works only for add, sub and mul.
There is a work in progress #13643 to provide broadcast support

[prajaramanTT]

@KalaivaniMCW @uazizTT Is this still an open issue ? If not, can you please close this ticket ? Thanks.

[uazizTT]

@KalaivaniMCW @uazizTT Is this still an open issue ? If not, can you please close this ticket ? Thanks.

I think we still need this issue to track support of broadcast across all binary ops. If someone has a list of ops that support broadcasting for both operands of a binary op then please update it here.

[KalaivaniMCW]

@uazizTT The broadcast support for binary operations has been added under binary_ng, the ops available as ttnn.experimental.add etc.
The list of binary ops with new broadcast support here
We are planning to integrate the functionality to existing binary op apis as well after profiling of the ops.
Could you test the shapes that failed earlier or share the shapes and op name here ?

In case of existing binary op apis, list of ops that support broadcasting for both operands are - ttnn.add/sub/mul

[uazizTT]

@KalaivaniMCW Thank you for updating about the progress, is there any other change in the function signatures besides adding experimental? If it just a name change, I can try to test it at our end.

For the shapes, you can try any number of arbitrary shapes. If you think those are working now, the next steps would be run tests from tt-mlir and tt-torch repositories. We need to update the documentation about the function signatures, but you can try to following the build steps to see if those tests are passing with these new operations.

[KalaivaniMCW]

@uazizTT yes.. there maybe some features that are still in progress, like sharding and then output_dtype typecasting. That is why these new ops are under ttnn.experimental - once they have all the eltwise features they will be integrated into the existing binary op APIs.
we don't expect everyone to switch to ttnn.experimental.op as it is temporary ( but if you need to test any cases that failed earlier, you can use the ttnn.experimental API to make sure we've given the support that is expected. ) soon these features will be integrated to the ttnn.op APIs.