Lowered hardswish

[lucasc896]

🚀 Feature

A lowered version of hardswish to enable performant use of torchvision.models.mobilenet_v3_small and other models on TPU

pt-xla-profiler: Op(s) not lowered: aten::hardswish_, aten::hardswish_backward,  Please open a GitHub issue with the above op lowering requests.

[JackCaoG]

Thanks for reporting, I will add this to our op lowering queue.

[lucasc896]

Would it be simple enough to replicate this PR, but with the function calculation changed? I have minimal experience with under the hood of this codebase, but I'm happy to give it a shot.

[JackCaoG]

it would be pretty similar to that I think. Some of the things in that pr is out of date (like we no longer has scripts/gen.py) but it mostly applied to what you want to do.

Please refer to https://github.com/pytorch/xla/blob/master/CONTRIBUTING.md for building pt/xla and https://github.com/pytorch/xla/blob/master/OP_LOWERING_GUIDE.md for how to lower an op. Happy to help if you run into issues.

[lucasc896]

Thanks for the pointers. I've got a branch ready to push, but I can't access the repo. I'm not sure whether the problem is on my side or not, so naive question: should I have push/write access to the repo by default?

[JackCaoG]

Ah, so since you do not have write access to PyTorch/XLA repo, what you want to do just fork a PyTorch/XLA repro, push a new branch to your fork, and open a pull request from you new branch to master.

Before you opening a pull requiest, also make sure to run the formater and build is passing. You can refer to https://github.com/pytorch/xla/blob/master/CONTRIBUTING.md#before-submiting-a-pull-request

[lucasc896]

I get the error: "No more forks can be created. View existing forks." Perhaps I can just use an existing fork?

[JackCaoG]

Seems like you already fork a pytorch/xla in https://github.com/lucasc896/xla

[wonjoolee95]

Closing as #3496 is merged. Thanks!