Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark results to docs #3244

Merged
merged 5 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions doc/htmldoc/benchmark_results.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
.. _nest_benchmark_results:

NEST performance benchmarks
===========================


NEST performance is continuously monitored and improved across various network sizes.
Here we show benchmarking results for NEST version 3.8 on Jureca-DC.
Copy link

@diesmann diesmann Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace: "for this version of NEST on JURECA-DC [x]."
Jureca is discribed in the large scale instrument Journal of Juelich. Please include the reference in the usual citation format. Here is the paper., it also has a doi: https://jlsrf.org/index.php/lsf/article/view/182/pdf
Add a sentence: "The benchmarking framework and the structure of the graphs is described in [Y]."
Give for [Y] the reference to the beNNch paper in the usual citation format:
Albers J, Pronold J, Kurth AC, Vennemo SB, Haghighi Mood K, Patronis A, Terhorst D, Jordan J, Kunkel S, Tetzlaff T, Diesmann M and Senk J (2022) A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations. Front. Neuroinform. 16:837549. doi: 10.3389/fninf.2022.837549



Strong scaling experiment of the Microcircuit model [1]_
---------------------------------------------------------

.. grid:: 1 1 1 1

.. grid-item::
:columns: 10
:class: sd-align-major-center

.. image:: /static/img/mc_benchmark.png

.. grid:: 1 1 1 1

.. grid-item::
:columns: 10
:class: sd-align-minor-center


* The model has ~80 000 neurons and ~300 million synapses
Copy link

@diesmann diesmann Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add: ", minimal delay 0.1ms"
add new bullet item:

  • 2 MPI processes per node, 64 threads per MPI process

* Increasing number of computing resources decrease simulation time

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add bullet item:

  • data averaged over 3 runs with different seeds, error bars indicate standard deviation

* The model runs faster than real time

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add reference [Z] at end of sentence so that the concept of "real time" can be looked up and there is a link for further information: Here is the reference:
Kurth, A. C. ; Senk, J.; Terhorst, D.; Finnerty, J.; Diesmann, M. (2022) Sub-realtime simulation of a neuronal network of natural density. Neuromorphic computing and engineering 2(2), 021001 (2022)
you can add doi link





Strong scaling experiment of the Multi-area-model [2]_
-------------------------------------------------------

.. grid:: 1 1 1 1

.. grid-item::
:class: sd-align-major-center
:columns: 10

.. image:: /static/img/mam_benchmark.png


.. grid:: 1 1 1 1

.. grid-item::
:columns: 10
:class: sd-align-minor-center

* The model has ~4.1 million neurons and ~24 billion synapses

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add: ", minimal delay 0.1ms"
add new bullet item:

2 MPI processes per node, 64 threads per MPI process

* Steady decrease of run time with additional compute resources

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add bullet item:

  • data averaged over 3 runs with different seeds, error bars indicate standard deviation





Weak scaling experiment of the HPC benchmark model [3]_
--------------------------------------------------------

.. grid:: 1 1 1 1

.. grid-item::
:columns: 10
:class: sd-align-major-center

.. image:: /static/img/hpc_benchmark.png


.. grid:: 1 1 1 1

.. grid-item::
:columns: 10
:class: sd-align-minor-center


* The size of network scales proportionally with the computational resources used

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add: ", minimal delay 1.5ms"
add new bullet item:

2 MPI processes per node, 64 threads per MPI process

* Largest network size in this diagram: ~5.8 million neurons and ~65 billion synapses
* The figure shows that NEST can handle massive networks and simulate them efficiently

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add bullet item:

  • data averaged over 3 runs with different seeds, error bars indicate standard deviation



.. seealso::

Example networks:

* :doc:`/auto_examples/Potjans_2014/index`
* `Multi-area model <https://inm-6.github.io/multi-area-model/>`_
* :doc:`/auto_examples/hpc_benchmark`

References
----------

.. [1] Potjans TC. and Diesmann M. 2014. The cell-type specific cortical
microcircuit: relating structure and activity in a full-scale spiking
network model. Cerebral Cortex. 24(3):785–806. DOI: `10.1093/cercor/bhs358 <https://doi.org/10.1093/cercor/bhs358>`__.


.. [2] Schmidt M, Bakker R, Hilgetag CC, Diesmann M and van Albada SJ. 2018. Multi-scale
account of the network structure of macaque visual cortex. Brain Structure
and Function. 223: 1409 https://doi.org/10.1007/s00429-017-1554-4

.. [3] Jordan J, Ippen T, Helias M, Kitayama I, Sato M, Igarashi J, Diesmann M, Kunkel S. 2018.
Extremely scalable spiking neuronal network simulation code: From laptops to exacale computers.
Frontiers in Neuroinformatics. 12. https://www.frontiersin.org/journals/neuroinformatics/articles/10.3389/fninf.2018.00002
6 changes: 6 additions & 0 deletions doc/htmldoc/hpc/benchmarking.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ Benchmarking NEST

When compiling NEST to perform benchmarks, see :ref:`our cmake options <performance_cmake>` for improved performance and energy saving.

NEST performance
~~~~~~~~~~~~~~~~

We regulary test NEST performance at various network sizes.
For results of these experiments see our :ref:`nest_benchmark_results` page.

beNNch
~~~~~~

Expand Down
8 changes: 6 additions & 2 deletions doc/htmldoc/hpc/optimizing_nest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ hardware and software of the HPC system you are using for improving the overall
performance.


Although there will be some variation between scripts and HPC systems, in general, we recommend that
Although there will be some variation between scripts and HPC systems, in general, we recommend that

* resources are fully utilized (e.g., all available cores are used).
* one thread pinned to one core (no simultaneous multithreading)
Expand Down Expand Up @@ -59,7 +59,11 @@ Although there will be some variation between scripts and HPC systems, in genera

.. seealso::

See our other guides on
See our other guides on

* :ref:`parallel_computing`
* :ref:`benchmark`

or see our performance results:

* :ref:`nest_benchmark_results`
1 change: 1 addition & 0 deletions doc/htmldoc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ How NEST works --- The Big Picture
Models <models/index>
Python API <ref_material/pynest_api/index>
ref_material/glossary
benchmark_results
Cite NEST <citing-nest>
License <license>

Expand Down
Binary file added doc/htmldoc/static/img/hpc_benchmark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/htmldoc/static/img/mam_benchmark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/htmldoc/static/img/mc_benchmark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading