How to run Cardinal on GPU? #802

tranm-ansto · 2023-10-17T06:05:54Z

tranm-ansto
Oct 17, 2023

Hello,

I'm trying to run the 'sfr_7pin' example (link below) on GPU.
https://github.com/neams-th-coe/cardinal/tree/devel/tutorials/sfr_7pin

I ran the example on CPU and it ran through fine. However, when I tried running on GPU, I got the error below. I'm new to Cardinal and NekRS. I'm not sure if there is additional stuff needed for running simulations on GPU. Thank you for your help.

Regards,
Minh

Terminal output below:

Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
In UnstructuredMesh::stitch_meshes:
This mesh has 6 nodes on boundary ' (30000). Other mesh has 6 nodes on boundary ' (31000).
Minimum edge length on both surfaces is 0.0011547.
In UnstructuredMesh::stitch_meshes:
Found 6 matching nodes.

In UnstructuredMesh::stitch_meshes:
This mesh has 6 nodes on boundary ' (30000). Other mesh has 6 nodes on boundary ' (31000).
Minimum edge length on both surfaces is 0.0011547.
In UnstructuredMesh::stitch_meshes:
Found 6 matching nodes.

             __    ____  _____

____ ___ / /__ / __ / /
/ __ \ / _ \ / //// // /_
/ / / // // ,< / , // /
// // __///||// ||/____/ v23.0.7 (907edeac)

MPI tasks: 8

Initializing device

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

MPI_ABORT was invoked on rank 4 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

*** ERROR ***

---[ Error ]--------------------------------------------------------------------
File : /data2/tranm/software/cardinal/contrib/nekRS/3rd_party/occa/src/occa/internal/modes/cuda/device.cpp
Line : 31
Function : device
Message : Device: Creating Device
CUDA Error [ 101 ]: CUDA_ERROR_INVALID_DEVICE
Stack
27 occa::error(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
26 occa::cuda::error(cudaError_enum, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
25 occa::cuda::device::device(occa::json const&)
24 occa::cuda::cudaMode::newDevice(occa::json const&)
23 occa::device::setup(occa::json const&)
22 occa::device::setup(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
21 device_t::device_t(setupAide&, comm_t&)
20 platform_t::platform_t(setupAide&, ompi_communicator_t, ompi_communicator_t)
19 nekrs::setup(ompi_communicator_t, ompi_communicator_t, int, int, int, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int)
18 NekInitAction::act()
17 Action::timedAct()
16 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
15 ActionWarehouse::executeAllActions()
14 MooseApp::runInputFile()
13 MultiApp::createApp(unsigned int, double)
12 MultiApp::createLocalApp(unsigned int)
11 MultiApp::createApps()
10 FEProblemBase::addMultiApp(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, InputParameters&)
9 Action::timedAct()
8 ActionWarehouse::executeActionsWithAction(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)
7 ActionWarehouse::executeAllActions()
6 MooseApp::runInputFile()
5 MooseApp::run()
4 /home/tranm/software/cardinal/cardinal-opt(main+0x551)
3 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)
2 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)
1 /home/tranm/software/cardinal/cardinal-opt(+0x104e5)

[moose106:526988] 6 more processes have sent help message help-mpi-api.txt / mpi-abort
[moose106:526988] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

aprilnovak · 2023-10-17T13:17:58Z

aprilnovak
Oct 17, 2023
Maintainer

Can you tell me more about. how you tried to run on GPU? You will need to change the command you use to launch Cardinal.

If you take a look at cardinal/scripts/job_summit, you'll see an example at the end of that file for the different commands we use to launch Cardinal on CPU vs. GPU.

13 replies

tranm-ansto Oct 25, 2023
Author

Thanks to the NekRS discussions page, I was able to get NekRS to run on GPU. The magic command is

mpirun --mca osc ucx -np 1 ~/software/cardinal/cardinal-opt -i nek.i

However, when I tried running coupled NekRS MOOSE, e.g., sfr_7pin, GPU works on NekRS part, but when Cardinal calls MOOSE, it seems MOOSE is only running on 1 CPU core. If I specify more cores -np 8, then the run crashes. I guess Cardinal is trying to run on 8 GPUs, but I only have 1.

Is there a way to have NekRS run on GPU and MOOSE run on CPU?

tranm-ansto Oct 27, 2023
Author

It turns out that if I just used Cardinal CPU and the command

mpirun -np 32 ~/software/cardinal/cardinal-opt -i solid.i --nekrs-backend CUDA --nekrs-device-id 0

Cardinal runs NekRS on the one GPU that I have and MOOSE on 32 CPU cores.

I'll close the tread.

aprilnovak Oct 27, 2023
Maintainer

Yep! Cardinal will always run MOOSE on the CPU, and NekRS on the GPU and facilitate the data transfers between them. I guess I'm still not sure on why you got an error with that command when using mpiexec -np 8 but not with mpirun -np 32 - because otherwise they are the same as an earlier post in this thread?

tranm-ansto Oct 28, 2023
Author

Sorry for not being clear.

When I install Cardinal with OCCA_CUDA_ENABLED=1 in the Makefile, i.e., Cardinal compiles for GPU, I can only run with
mpirun --mca osc ucx -np 1 ~/software/cardinal/cardinal-opt -i solid.i
Only 1 GPU and 1 CPU (I only have 1 GPU card on the machine that I'm testing). Anything other than that would not run.

So, I clean out Cardinal and reinstall without modifying the Makefile, i.e., Cardinal does CPU build by default. The command
mpirun -np 32 ~/software/cardinal/cardinal-opt -i solid.i --nekrs-backend CUDA --nekrs-device-id 0
runs the sfr_7pin problem on 32 CPU and 1 GPU.

If I trympirun -np 64 (64 CPU and 1 GPU), I get a message with CUDA Error [ 2 ]: CUDA_ERROR_OUT_OF_MEMORY. The GPU only has 24GB. I guess it got something to do with memory distribution.

The gas_compact_cht problem does not even run on 1 CPU and 1 GPU. I get the message CUDA Error [ 2 ]: CUDA_ERROR_OUT_OF_MEMORY pretty much right away. I guess the problem is too big to load into GPU?

I guess we will need to upgrade our GPUs to take advantage of the GPU capability of NekRS.

aprilnovak Oct 28, 2023
Maintainer

To run Cardinal on GPU, you'll always need to have one of those OCCA_<backend>_ENABLED=1 set in the Makefile (you might have done this, but I was not 100% clear on your wording).

The gas_compact_cht case is quite large -- it will not fit on 1 GPU. That case has a relatively high Reynolds number (~200,000) and it is wall-resolved, so the mesh is very fine. I recall that it did not fit on a CPU system I had access to with 1 TB RAM - it requires HPC.

The sfr_7pin case is very small, and will work for testing on your 1 GPU.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to run Cardinal on GPU? #802

{{title}}

Replies: 1 comment 13 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

How to run Cardinal on GPU? #802

tranm-ansto Oct 17, 2023

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes. You may or may not see output from other processes, depending on exactly when Open MPI kills them.

Replies: 1 comment · 13 replies

aprilnovak Oct 17, 2023 Maintainer

tranm-ansto Oct 25, 2023 Author

tranm-ansto Oct 27, 2023 Author

aprilnovak Oct 27, 2023 Maintainer

tranm-ansto Oct 28, 2023 Author

aprilnovak Oct 28, 2023 Maintainer

tranm-ansto
Oct 17, 2023

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

Replies: 1 comment 13 replies

aprilnovak
Oct 17, 2023
Maintainer

tranm-ansto Oct 25, 2023
Author

tranm-ansto Oct 27, 2023
Author

aprilnovak Oct 27, 2023
Maintainer

tranm-ansto Oct 28, 2023
Author

aprilnovak Oct 28, 2023
Maintainer