Fast Multi-ahead Attention support on AMD ROCM #393

Job	Run time
build (8-core-ubuntu, 3.10, 2.2.0, 118) / ubuntu-py3.10-pt2.2.0+cu118	1h 38m 5s
build (8-core-ubuntu, 3.10, 2.2.0, 121) / ubuntu-py3.10-pt2.2.0+cu121	1h 15m 18s
build (8-core-ubuntu, 3.11, 2.2.0, 118) / ubuntu-py3.11-pt2.2.0+cu118	1h 37m 15s
build (8-core-ubuntu, 3.11, 2.2.0, 121) / ubuntu-py3.11-pt2.2.0+cu121	1h 11m 47s
build (8-core-ubuntu, 3.8, 2.2.0, 121) / ubuntu-py3.8-pt2.2.0+cu121	1h 13m 26s
build (8-core-ubuntu, 3.8, 2.2.0, 118) / ubuntu-py3.8-pt2.2.0+cu118	1h 36m 31s
build (8-core-ubuntu, 3.9, 2.2.0, 121) / ubuntu-py3.9-pt2.2.0+cu121	1h 13m 45s
build (8-core-ubuntu, 3.9, 2.2.0, 118) / ubuntu-py3.9-pt2.2.0+cu118	1h 37m 25s
build (windows-8-core, 3.10, 2.2.0, 118) / win-py3.10-pt2.2.0+cu118	36m 55s
build (windows-8-core, 3.10, 2.2.0, 121) / win-py3.10-pt2.2.0+cu121	15m 43s
build (windows-8-core, 3.11, 2.2.0, 118) / win-py3.11-pt2.2.0+cu118	38m 7s
build (windows-8-core, 3.11, 2.2.0, 121) / win-py3.11-pt2.2.0+cu121	19m 11s
build (windows-8-core, 3.8, 2.2.0, 121) / win-py3.8-pt2.2.0+cu121	19m 47s
build (windows-8-core, 3.8, 2.2.0, 118) / win-py3.8-pt2.2.0+cu118	37m 36s
build (windows-8-core, 3.9, 2.2.0, 121) / win-py3.9-pt2.2.0+cu121	15m 31s
build (windows-8-core, 3.9, 2.2.0, 118) / win-py3.9-pt2.2.0+cu118	36m 14s
upload_pt_cu118	0s
upload_pip	0s
upload_pt_cu121	0s
	15h 2m 36s

Provide feedback