Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark results to docs #3244

Merged
merged 5 commits into from
Jul 22, 2024

Conversation

jessica-mitchell
Copy link
Contributor

@jessica-mitchell jessica-mitchell commented Jul 2, 2024

This PR adds doc page that showcases the performance results of different scaling experiments.

  • NOTE: a temp image is used and needs to be replaced with correct images once the benchmarks run on rc_3.8!
    images should be stored in /doc/htmldoc/static/img/

Thanks @ackurth and s. theinen for your work on this

Fixes #3243

Read the Docs

https://nest-simulator--3244.org.readthedocs.build/en/3244/benchmark_results.html

@jessica-mitchell jessica-mitchell added T: Enhancement New functionality, model or documentation S: High Should be handled next I: No breaking change Previously written code will work as before, no one should note anything changing (aside the fix) labels Jul 2, 2024
@jessica-mitchell jessica-mitchell added this to the NEST 3.8 milestone Jul 2, 2024
@jessica-mitchell jessica-mitchell requested a review from mlober July 22, 2024 09:43
doc/htmldoc/benchmark_results.rst Outdated Show resolved Hide resolved
doc/htmldoc/benchmark_results.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@mlober mlober left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these updates.

@jessica-mitchell jessica-mitchell merged commit 0d0b0a8 into nest:master Jul 22, 2024
24 checks passed


NEST performance is continuously monitored and improved across various network sizes.
Here we show benchmarking results for NEST version 3.8 on Jureca-DC.
Copy link

@diesmann diesmann Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace: "for this version of NEST on JURECA-DC [x]."
Jureca is discribed in the large scale instrument Journal of Juelich. Please include the reference in the usual citation format. Here is the paper., it also has a doi: https://jlsrf.org/index.php/lsf/article/view/182/pdf
Add a sentence: "The benchmarking framework and the structure of the graphs is described in [Y]."
Give for [Y] the reference to the beNNch paper in the usual citation format:
Albers J, Pronold J, Kurth AC, Vennemo SB, Haghighi Mood K, Patronis A, Terhorst D, Jordan J, Kunkel S, Tetzlaff T, Diesmann M and Senk J (2022) A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations. Front. Neuroinform. 16:837549. doi: 10.3389/fninf.2022.837549

:class: sd-align-minor-center


* The model has ~80 000 neurons and ~300 million synapses
Copy link

@diesmann diesmann Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add: ", minimal delay 0.1ms"
add new bullet item:

  • 2 MPI processes per node, 64 threads per MPI process


* The model has ~80 000 neurons and ~300 million synapses
* Increasing number of computing resources decrease simulation time
* The model runs faster than real time

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add reference [Z] at end of sentence so that the concept of "real time" can be looked up and there is a link for further information: Here is the reference:
Kurth, A. C. ; Senk, J.; Terhorst, D.; Finnerty, J.; Diesmann, M. (2022) Sub-realtime simulation of a neuronal network of natural density. Neuromorphic computing and engineering 2(2), 021001 (2022)
you can add doi link

:columns: 10
:class: sd-align-minor-center

* The model has ~4.1 million neurons and ~24 billion synapses

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add: ", minimal delay 0.1ms"
add new bullet item:

2 MPI processes per node, 64 threads per MPI process

:class: sd-align-minor-center


* The size of network scales proportionally with the computational resources used

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add: ", minimal delay 1.5ms"
add new bullet item:

2 MPI processes per node, 64 threads per MPI process



* The model has ~80 000 neurons and ~300 million synapses
* Increasing number of computing resources decrease simulation time

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add bullet item:

  • data averaged over 3 runs with different seeds, error bars indicate standard deviation

:class: sd-align-minor-center

* The model has ~4.1 million neurons and ~24 billion synapses
* Steady decrease of run time with additional compute resources

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add bullet item:

  • data averaged over 3 runs with different seeds, error bars indicate standard deviation


* The size of network scales proportionally with the computational resources used
* Largest network size in this diagram: ~5.8 million neurons and ~65 billion synapses
* The figure shows that NEST can handle massive networks and simulate them efficiently

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add bullet item:

  • data averaged over 3 runs with different seeds, error bars indicate standard deviation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I: No breaking change Previously written code will work as before, no one should note anything changing (aside the fix) S: High Should be handled next T: Enhancement New functionality, model or documentation
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add benchmarking diagrams to docs
4 participants