Tribits: find SYCL libraries for TPL MKL #13520
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brian-kelley
added
type: enhancement
Signed-off-by: Brian Kelley <[email protected]>
brian-kelley
force-pushed
the
oneapi_mkl
branch
from
df93792
to
0aee2ae
Compare
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: brian-kelley |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
lucbv
approved these changes
Oct 10, 2024
|
Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ lucbv ]! |
|
Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR... |
bartlettroscoe
approved these changes
Oct 10, 2024
There was a problem hiding this comment.
By avoiding changing FindTPLBLAS.cmake and FindTPLLAPACK.cmake, and having these changes only kick in when Kokkos_ENABLE_SYCL is set to true, this is a very safe and straightforward PR. Looks good to merge to me.
(NOTE: This kicks the can down the road w.r.t. to getting BLAS and LAPACK automatically when MKL is enabled and found , but we can't let "best" be the enemy of "better", and incremental is better than all-at-once.)
|
Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR... |
|
@lucbv I agree, there isn't a |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@trilinos/tribits
@trilinos/kokkos-kernels
Motivation
This PR allows OneAPI/MKL with SYCL support to be enabled in Trilinos, where it wasn't before. With MKL+SYCL enabled, KokkosKernels can call MKL kernels on both host and device.
We now have a nightly performance build on Sunspot, but it can't enable MKL because of this problem.
The only caveat is SYCL+MKL requires OneAPI 2024.1 or above, since before that, the SYCL libraries were not available for the intel_lp64 interface that Trilinos needs. intel_lp64 means that BLAS/LAPACK functions use int (32-bit) argument types for dimensions and strides. This is required for when the Teuchos BLAS and LAPACK wrappers call functions from MKL libs.
Related Issues
Partly supersedes #13330. This is only about enabling MKL for SYCL GPUs. This doesn't attempt to explicitly define the BLAS and LAPACK TPLs in terms of MKL. Nothing changes for users of MKL on CPU; they are still responsible for pointing CMake to the BLAS/LAPACK libraries they want (whether that is also MKL or something else).
That is still worthwhile but as @bartlettroscoe said in the #13330 discussion, it:
a) requires extra care to have Tribits install the correct BLASConfig and LAPACKConfig cmake files, and
b) may break numerical backwards compatibility for users that loaded both MKL and non-MKL BLAS/LAPACK.
Stakeholder Feedback
Testing
Tested on Sunspot with MKL on GPU and CPU, and on Blake with MKL on CPU only. KokkosKernels and Tpetra tests passed. On Sunspot I used the kernel logger to verify that the OneAPI was being used by KokkosKernels.