Vega/RocM support status

[yupinov]

Hi,
I'm working on OpenCL implementation of Gromacs (https://github.com/gromacs/gromacs). What is needed for one of the algorithms is 3D real to/from hermitian interleaved FFT. I got it to work on older AMD/NVIDIA hardware, but on Vega with RocM stack it produces garbage data. Is this expected/known to not work?

clinfo

Platform Name AMD Accelerated Parallel Processing
Number of devices 1
Device Name gfx900
Device Vendor Advanced Micro Devices, Inc.
Device Vendor ID 0x1002
Device Version OpenCL 1.2
Driver Version 2576.0 (HSA1.1,LC)
Device OpenCL C Version OpenCL C 2.0
Device Type GPU
Device Profile FULL_PROFILE
Device Board Name (AMD) Vega 10 XTX [Radeon Vega Frontier Edition]

Test -s --version

clFFT client API version: 2.15.0
clFFT runtime version: 2.15.0
...

[==========] 882 tests from 4 test cases ran. (419095 ms total)
[ FAILED ] 166 tests, listed below:
[ FAILED ] accuracy_test_pow2_single.large_1D_in_place_real_to_hermitian_interleaved
[ FAILED ] accuracy_test_pow2_single.large_1D_4M_in_place_real_to_hermitian_interleaved
[ FAILED ] accuracy_test_pow2_single.large_1D_out_of_place_real_to_hermitian_interleaved
[ FAILED ] accuracy_test_pow2_single.large_1D_out_of_place_real_to_hermitian_planar
[ FAILED ] accuracy_test_pow2_single.normal_2D_in_place_real_to_hermitian_interleaved
[ FAILED ] accuracy_test_pow2_single.normal_2D_in_place_hermitian_interleaved_to_real
[ FAILED ] accuracy_test_pow2_single.normal_2D_out_of_place_real_to_hermitian_interleaved
... (and so on, seemingly mostly non-C2C tests)
166 FAILED TESTS

[yupinov]

Just tried same short test run with Fiji card and fresh Rocm 1.8.0 (driver 2617), still 171/882 tests failed.

[pszi1ard]

Ping! Can anyone give an update on this?

[bragadeesh]

Answered here: ROCm/rocFFT#120

[pszi1ard]

@bragadeesh: looks like the ROCm 1.9 release fixed those cases that we care about. Thanks for that!

Note that there are still four tests that fail, all double precision:

 $ tail -n10 test-dev-gpu02.log
[  PASSED  ] 16436 tests.
[  FAILED  ] 4 tests, listed below:
[  FAILED  ] mixed_radices/mixed_radix.double_precision_hermitian_to_real_auto_generated/173, where GetParam() = 70
[  FAILED  ] clfft_DirectedTest_double_1d_inv/accuracy_test_directed_real.inplace_interleaved/47, where GetParam() = 120-byte object <13-00 00-00 00-00 00-00 02-00 00-00 01-00 00-00 01-00 00-00 00-00 00-00 B0-FD 83-68 3B-7F 00-00 B8-FD 83-68 3B-7F 00-00 B8-FD 83-68 3B-7F 00-00 D0-FD 83-68 3B-7F 00-00 D8-FD 83-68 3B-7F 00-00 D8-FD 83-68 3B-7F 00-00 F0-FD 83-68 3B-7F 00-00 F8-FD 83-68 3B-7F 00-00 F8-FD 83-68 3B-7F 00-00 24-00 00-00 00-00 00-00 48-00 00-00 00-00 00-00 03-00 00-00 05-00 00-00>
[  FAILED  ] mixed_radices_postcallback/mixed_radix_postcallback.double_precision_complex_to_complex_auto_generated/227, where GetParam() = 490
[  FAILED  ] mixed_radices_postcallback/mixed_radix_postcallback.double_precision_hermitian_to_real_auto_generated/173, where GetParam() = 70

 4 FAILED TESTS
  YOU HAVE 299 DISABLED TESTS

[bragadeesh]

thanks, good to see you are able to see progress; compiler team on our side looking at these

[pszi1ard]

Just ran the clFFT tests and with ROCm 2.3 I get all tests pass on both Fiji and Vega.

 tail test-dev-gpu02.log 
1 warning generated.
[       OK ] mixed_radices_postcallback/mixed_radix_postcallback.double_precision_real_to_hermitian_auto_generated/247 (574 ms)
[----------] 1488 tests from mixed_radices_postcallback/mixed_radix_postcallback (2030325 ms total)

[----------] Global test environment tear-down
[==========] 16440 tests from 46 test cases ran. (30218991 ms total)
[  PASSED  ] 16440 tests.

  YOU HAVE 299 DISABLED TESTS

Good job!

[pszi1ard]

PS: If tests passing can be consider "support complete" than this can be closed, but I'd be glad to see some activity around getting good performance too.