diff --git a/images/summit-node-1rs-1task-1gpu-example.png b/images/summit-node-1rs-1task-1gpu-example.png deleted file mode 100644 index 33353275..00000000 Binary files a/images/summit-node-1rs-1task-1gpu-example.png and /dev/null differ diff --git a/images/titan-node-1task-1gpu.png b/images/titan-node-1task-1gpu.png deleted file mode 100644 index d29f6041..00000000 Binary files a/images/titan-node-1task-1gpu.png and /dev/null differ diff --git a/systems/summit_user_guide.rst b/systems/summit_user_guide.rst index 0a12fc4b..1aba08ac 100644 --- a/systems/summit_user_guide.rst +++ b/systems/summit_user_guide.rst @@ -2573,81 +2573,6 @@ It's recommended to explicitly specify ``jsrun`` options. This most often includes ``--nrs``,\ ``--cpu_per_rs``, ``--gpu_per_rs``, ``--tasks_per_rs``, ``--bind``, and ``--launch_distribution``. -Aprun to jsrun -"""""""""""""" - -Mapping aprun commands used on Titan to Summit's jsrun is only possible -in simple single GPU cases. The following table shows some basic single -GPU examples that could be executed on Titan or Summit. In the single -node examples, each resource set will resemble a Titan node containing a -single GPU and one or more cores. Although not required in each case, -common jsrun flags (resource set count, GPUs per resource set, tasks per -resource set, cores per resource set, binding) are included in each -example for reference. The jsrun ``-n`` flag can be used to increase the -number of resource sets needed. Multiple resource sets can be created on -a single node. If each MPI task requires a single GPU, up to 6 resource -sets could be created on a single node. - -+-------------------------+-------------+--------------------+-----------------+-------------------------------------+ -| GPUs per Resource Set | MPI Tasks | Threads per Task | aprun | jsrun | -+=========================+=============+====================+=================+=====================================+ -| 1 | 1 | 0 | aprun -n1 | jsrun -n1 -g1 -a1 -c1 | -+-------------------------+-------------+--------------------+-----------------+-------------------------------------+ -| 1 | 2 | 0 | aprun -n2 | jsrun -n1 -g1 -a2 -c1 | -+-------------------------+-------------+--------------------+-----------------+-------------------------------------+ -| 1 | 1 | 4 | aprun -n1 -d4 | jsrun -n1 -g1 -a1 -c4 -bpacked:4 | -+-------------------------+-------------+--------------------+-----------------+-------------------------------------+ -| 1 | 2 | 8 | aprun -n2 -d8 | jsrun -n1 -g1 -a2 -c16 -bpacked:8 | -+-------------------------+-------------+--------------------+-----------------+-------------------------------------+ - -The jsrun ``-n`` flag can be used to increase the number of resource -sets needed. Multiple resource sets can be created on a single node. If -each MPI task requires a single GPU, up to 6 resource sets could be -created on a single node. - -For cases when the number of tasks per resource set (i.e. the ``-a`` -flag) is greater than one, the job must use ``-alloc_flags "gpumps"``. -This allows multiple tasks to share the same GPU. - -The following example images show how a single-gpu/single-task job -would be placed on a single Titan and Summit node. On Summit, the red -box represents a resource set created by jsrun. The resource set looks -similar to a Titan node, containing a single GPU, a single core, and -memory. - -+--------------+-------------------------+ -| Titan Node | Summit Node | -+==============+=========================+ -| aprun -n1 | jsrun -n1 -g1 -a1 -c1 | -+--------------+-------------------------+ -| |image18| | |image19| | -+--------------+-------------------------+ - -.. |image18| image:: /images/titan-node-1task-1gpu.png - :class: normal aligncenter -.. |image19| image:: /images/summit-node-1rs-1task-1gpu-example.png - :class: normal aligncenter - -Because Summit's nodes are much larger than Titan's were, 6 single-gpu -resource sets can be created on a single Summit node. The following -image shows how six single-gpu, single-task resource sets would be -placed on a node by default. In the example, the command -``jsrun -n6 -g1 -a1 -c1`` is used to create six single-gpu resource sets -on the node. Each resource set is indicated by differing colors. Notice, -the ``-n`` flag is all that changed between the above single resource -set example. The ``-n`` flag tells jsrun to create six resource sets. - -.. figure:: https://www.olcf.ornl.gov/wp-content/uploads/2018/03/summit-2node-1taskpergpu.png - :align: center -  - ``jsrun -n 6 -g 1 -a 1 -c 1`` starts 6 resource sets, each indicated by - differing colors. Each resource contains 1 GPU, 1 Core, and memory. The - red resource set contains GPU 0 and Core 0. The purple resource set - contains GPU 3 and Core 84. ``-n 6`` tells jsrun how many resource sets to - create. In this example, each resource set is similar to a single Titan - node. - - jsrun Examples ^^^^^^^^^^^^^^