additional solana-sys-tuner limits #22566
Conversation
|
If these values stick, please add them to the manual tuning section in the docs as well: https://docs.solana.com/running-validator/validator-start#manual |
Codecov Report
@@ Coverage Diff @@
## master #22566 +/- ##
=======================================
Coverage 81.1% 81.1%
=======================================
Files 560 561 +1
Lines 151206 151286 +80
=======================================
+ Hits 122633 122719 +86
+ Misses 28573 28567 -6 |
sys-tuner/src/main.rs
Outdated
| @@ -93,6 +93,10 @@ fn tune_kernel_udp_buffers_and_vmmap() { | |||
|
|
|||
| // increase mmap counts for many append_vecs | |||
| sysctl_write("vm.max_map_count", "1000000"); | |||
|
|
|||
| // Reference: https://community.mellanox.com/s/article/linux-sysctl-tuning | |||
There was a problem hiding this comment.
This document is referencing TCP tuning, so I poked around for the same vars WRT UDP and found https://indico.cern.ch/event/212228/contributions/1507212/attachments/333941/466017/10GE_network_tests_with_UDP.pdf. Admittedly it's tuning 10Gbit and using larger MTU, but at least the right protocol. The suggestions are quite a bit different and also tough a couple other knobs that we haven't investigated.
Additionally, it got me wondering whether we might be pinning PoH to the NIC's IRQ affinity core, which would probably be bad.
WDYT about rolling new values out to our nodes for a week and monitoring before we change the code?
There was a problem hiding this comment.
Yeah, let's try on our nodes first.
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
stale
bot
added
the
stale
|
This stale pull request has been automatically closed. Thank you for your contributions. |
|
Hey @jbiseda @t-nelson I am asking if we should change the zero value to something worthable. |
Problem
Add additional OS network tuning for burst traffic
Summary of Changes
Add
solana-sys-tunerupdated limits for:net.core.optmem_max=4194304
net.core.netdev_max_backlog=250000
per: https://community.mellanox.com/s/article/linux-sysctl-tuning
Fixes #