STK: Don't use `MPI_COMM_WORLD` #12167

JacobDomagala · 2023-08-24T12:26:42Z

@trilinos/stk

This PR is part of the initiative to phase out the use of MPI_COMM_WORLD in Trilinos.

trilinos-autotester · 2023-08-24T13:07:25Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

alanw0

Thanks for the proposed changes.

stk is developed in a separate repository and periodically snapshotted into trilinos, so we will need to make changes there to avoid having these get lost at the next snapshot.

We agree with the general premise that MPI_COMM_WORLD should not be used except in main and in functions that are equivalent to main such as gtest unit-tests.

We will prefer to implement some of these changes slightly differently, such as adding a mechanism to pass a communicator from a scope that already has one, rather than creating a global-comm facility.

What time-frame are you hoping to have MPI_COMM_WORLD usage eliminated in trilinos?

ccober6 · 2023-08-24T16:34:01Z

@alanw0, this is some contract work we are having done. They are hoping to complete this task by next week. :) If you can meet that timeline, it would be great, but not necessary. Otherwise, would you be able to get this in before the end of the FY (~Sep. 22) and the next release (Trilinos 15.0)?

Another part of this effort is to have a non-blocking PR test to warn developers about adding back MPI_COMM_WORLD usage, as a few years ago we had a similar effort and since have had some regression. :(

trilinos-autotester · 2023-08-24T18:03:32Z

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; This inspection will remain valid until a new commit to source branch is performed.

alanw0 · 2023-08-24T18:03:52Z

@ccober6 Thanks Curt, these changes don't look too difficult for stk, we can probably have them in stk/sierra by next week. Getting the snapshot back to trilinos is sometimes unpredictable. But beating the next release should not be a problem.

trilinos-autotester · 2023-08-24T18:05:20Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Build Num: 2711
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Build Num: 1231
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Build Num: 1221
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Build Num: 1220
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 2436
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	ascic
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Build Num: 2223
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	GPU
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_intel-2021.3

Build Num: 862
Status: STARTED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Using Repos:

Repo: TRILINOS (NexGenAnalytics/Trilinos)

Pull Request Author: JacobDomagala

ccober6 · 2023-08-24T18:11:28Z

Great! Thanks @alanw0!

trilinos-autotester · 2023-08-24T18:54:49Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Build Num: 2711
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Build Num: 1231
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-v2-gnu-8.3.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Build Num: 1221
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-8.3.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Build Num: 1220
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-clang-11.0.1-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_python3

Build Num: 2436
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	ascic
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Build Num: 2223
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	GPU
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

Build Information

Test Name: Trilinos_PR_intel-2021.3

Build Num: 862
Status: PASSED

Jenkins Parameters

Parameter Name	Value
FORCE_CLEAN	true
GENCONFIG_BUILD_NAME	rhel7_sems-intel-2021.3-sems-openmpi-4.0.5_release-debug_shared_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-off_no-package-enables
PR_LABELS
PULLREQUESTNUM	12167
PULLREQUEST_CDASH_TRACK	Pull Request
TEST_REPO_ALIAS	TRILINOS
TRILINOS_NODE_LABEL	trilinos-any
TRILINOS_SOURCE_BRANCH	nga-fy23-pr-candidate-9
TRILINOS_SOURCE_REPO	https://github.com/NexGenAnalytics/Trilinos
TRILINOS_SOURCE_SHA	`f6b9a1a`
TRILINOS_SRN_CONFIG	true
TRILINOS_TARGET_BRANCH	develop
TRILINOS_TARGET_REPO	https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA	`db9c884`

CDash Test Results for PR# 12167.

trilinos-autotester · 2023-08-24T18:55:07Z

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS BEEN REVIEWED, BUT NOT ACCEPTED OR REQUIRES CHANGES

trilinos-autotester · 2023-08-24T18:55:13Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

trilinos-autotester · 2023-08-25T19:19:19Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

trilinos-autotester · 2023-08-26T19:30:08Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

trilinos-autotester · 2023-08-27T19:30:10Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

jwillenbring · 2023-09-07T18:06:25Z

@alanw0 Please confirm when you have made (the equivalent of) these changes to STK. We can close this PR at that time.

alanw0 · 2023-09-07T18:42:54Z

@jwillenbring we have made most of them, still a couple of stragglers we should have in the next week. I agree it's probably safe to go ahead and close this. We'll have a new stk snapshot coming into trilinos soon also.

jwillenbring · 2023-11-13T18:10:28Z

@alanw0 Have these changes been made in the Trilinos repo?

alanw0 · 2023-11-13T18:53:53Z

@alanw0 Have these changes been made in the Trilinos repo?

No not yet. I have the changes in a snapshot of stk that I'm trying to put into trilinos, but it hasn't made it through PR testing yet. We apparently have something wrong with one of our newer doc-test executables. I'll try to get that through soon.

alanw0 · 2023-11-19T17:05:50Z

@jwillenbring these stk changes did go into trilinos/develop mid-late last week.

STK: Don't use MPI_COMM_WORLD

f6b9a1a

JacobDomagala self-assigned this Aug 24, 2023

JacobDomagala mentioned this pull request Aug 24, 2023

List of PRs to Trilinos/develop and fork development NexGenAnalytics/Trilinos#77

Open

stmcgovern added the AT: PRE-TEST INSPECTED Required to test outside contributions. This label alone will not allow a PR to merge. label Aug 24, 2023

alanw0 self-requested a review August 24, 2023 15:37

alanw0 reviewed Aug 24, 2023

View reviewed changes

trilinos-autotester removed the AT: PRE-TEST INSPECTED Required to test outside contributions. This label alone will not allow a PR to merge. label Aug 24, 2023

cgcgcg added the pkg: STK label Aug 24, 2023

alanw0 added the AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) label Aug 27, 2023

alanw0 closed this Nov 19, 2023

STK: Don't use MPI_COMM_WORLD #12167

STK: Don't use MPI_COMM_WORLD #12167

Conversation

JacobDomagala commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

alanw0 left a comment

Choose a reason for hiding this comment

ccober6 commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

alanw0 commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_python3

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_intel-2021.3

Jenkins Parameters

Using Repos:

ccober6 commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-serial

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_gcc-8.3.0-debug

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_clang-11.0.1

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_python3

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

Jenkins Parameters

Build Information

Test Name: Trilinos_PR_intel-2021.3

Jenkins Parameters

trilinos-autotester commented Aug 24, 2023

trilinos-autotester commented Aug 24, 2023

trilinos-autotester commented Aug 25, 2023

trilinos-autotester commented Aug 26, 2023

trilinos-autotester commented Aug 27, 2023

jwillenbring commented Sep 7, 2023

alanw0 commented Sep 7, 2023

jwillenbring commented Nov 13, 2023

alanw0 commented Nov 13, 2023

alanw0 commented Nov 19, 2023

STK: Don't use `MPI_COMM_WORLD` #12167

STK: Don't use `MPI_COMM_WORLD` #12167