running multiple threads on nekRS21.1

[chanleonzen]

Hello there,

I've managed to compile nekRS 21.1 on the HPC but it seems like 1 CPU is tied to 1GPU. In the HPC that I am using, each node has 48 CPUs and 4 GPUs. Ideally, I would like to utilise the 12CPUs when running on each GPU.

In my PBS script, I use the following lines for the turbPipePeriodic test case.

export OMP_NUM_THREADS=12
mpirun -np $PBS_NGPUS --map-by ppr:1:numa:PE=12 $NEKRS_HOME/bin/nekrs --setup turbPipe

Below is a snippet of the log file which shows that I am using 1 thread and 2 GPUs.

Is it possible to make it multithreaded? I know this is not ideal but I need to run some postprocessing in the usrchk() subroutine which utilises the CPUs.

Thank you.

Kind regards,
Leon

[stgeke]

The nekRS host code + nek5000 does not use/support openMP.

[chanleonzen]

Thank you for your clarification.