Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug Report] Ttnn::multiply does not broadcast in second dimension #13646

Closed
Tracked by #13795
dmakoviichuk-tt opened this issue Oct 9, 2024 · 3 comments
Closed
Tracked by #13795
Assignees
Labels
bug Something isn't working op_cat: eltwise P0

Comments

@dmakoviichuk-tt
Copy link
Contributor

Describe the bug
Ttnn::multiply do not broadcast in second dimension.
Fails silently by throwing near NaN/Inf valuesTo Reproduce
Steps to reproduce the behavior:

with ttnn.manage_device(device_id=0) as device:
    x = torch.ones((16, 16, 32, 32), dtype=torch.bfloat16)
    y = torch.ones((16, 1, 32, 32), dtype=torch.bfloat16) * 0.5
    
    x_tt = ttnn.from_torch(x, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)
    y_tt = ttnn.from_torch(y, dtype=ttnn.bfloat16, layout=ttnn.TILE_LAYOUT, device=device)

    x_y_mult_tt = ttnn.multiply(x_tt, y_tt)
    print(ttnn.to_torch(x_y_mult_tt))

It will print a garbage.
Expected behavior
it should work.

Additional context
Currently it requires us to generate broadcasted vectors on cpu.
It is a huge problem for a training performance.
@eyonland if you don't think you can fix it please let me know I'll find another owner.

@dmakoviichuk-tt dmakoviichuk-tt added bug Something isn't working P0 labels Oct 9, 2024
@dmakoviichuk-tt dmakoviichuk-tt changed the title [Bug Report] Ttnn::multiply do not broadcast in second dimension. [Bug Report] Ttnn::multiply does not broadcast in second dimension Oct 9, 2024
@eyonland
Copy link
Contributor

eyonland commented Oct 9, 2024

Are you able to use ttnn.bcast here instead as a workaround?

@umadevimcw
Copy link
Contributor

umadevimcw commented Oct 10, 2024

@dmakoviichuk-tt @eyonland #13673

The above PR holds functionality support for broadcasting the second dimension using repeat. This will affect the performance as we are using the repeat op but we can use it now for functionality and update this later once broadcast support is updated

umadevimcw added a commit that referenced this issue Oct 16, 2024
umadevimcw added a commit that referenced this issue Oct 16, 2024
umadevimcw added a commit that referenced this issue Oct 16, 2024
umadevimcw added a commit that referenced this issue Oct 16, 2024
umadevimcw added a commit that referenced this issue Oct 21, 2024
umadevimcw added a commit that referenced this issue Oct 22, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
umadevimcw added a commit that referenced this issue Oct 25, 2024
* #13646: Add second dimension support in binary op

* #13646: Add unit test for channel bcast using repeat

* #13646: Add assert statement

* #13646: Fix CI failure issue
@umadevimcw
Copy link
Contributor

PR is merged to main. Hence closing this issue

ct-clmsn pushed a commit to ct-clmsn/tt-metal that referenced this issue Nov 12, 2024
* tenstorrent#13646: Add second dimension support in binary op

* tenstorrent#13646: Add unit test for channel bcast using repeat

* tenstorrent#13646: Add assert statement

* tenstorrent#13646: Fix CI failure issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working op_cat: eltwise P0
Projects
None yet
Development

No branches or pull requests

3 participants