-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Networking] set TCP buffer overrides to None by default #10176
Conversation
Forge result links:
|
} else { | ||
4500 | ||
}; | ||
let min_expected_tps = 4500; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping this pretty conservative for now. I've seen a range of 5-6.2K TPS.
@@ -157,10 +149,6 @@ impl NetworkConfig { | |||
inbound_rate_limit_config: None, | |||
outbound_rate_limit_config: None, | |||
max_message_size: MAX_MESSAGE_SIZE, | |||
inbound_rx_buffer_size_bytes: Some(INBOUND_TCP_RX_BUFFER_SIZE), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Silly question, but, instead of removing support for this entirely, is there a way to just set the default to not override the values (i.e., default to None
in the config and have the code do nothing)? e.g., I'm just wondering if there's ever a scenario where we want to set these things?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re-adding tcp socket option setting would have the annoying work of figuring out where it goes, so leaving this on but disabled (it already had Option if-None-do-nothing) makes sense to me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I honestly don't think we want to mess with this and so might as well clean up the configs now. For this to be effective the user would have to bump up the linux configs, otherwise it has a detrimental effect.
If you feel strongly I'll leave the config but add a comment on the ways it shouldn't be used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leave it in or at least leave a comment // TODO: if needed, set connection socket options here
; someone is going to want to tinker with this again, even if Linux has pretty good defaults
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @JoshLind @brianolson . Makes sense, I guess there are other socket options that someone might want to play with in the future as well. I've changed the PR to just set the defaults as None
and put up a big comment to proceed with caution.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
✅ Forge suite
|
✅ Forge suite
|
✅ Forge suite
|
Tests show that overriding the socket options for send and recv buffer actually hurts performance (even with default linux kernel settings). Logically, this rolls back #4649 but keeping the configs and code around in case this or other socket options are revisited. New results: * `net_bench_two_region_env`: 800 KB/s -> 4 MB/s * `pfn_performance_with_realistic_env`: 4K -> 6K TPS
…0775) Tests show that overriding the socket options for send and recv buffer actually hurts performance (even with default linux kernel settings). Logically, this rolls back #4649 but keeping the configs and code around in case this or other socket options are revisited. New results: * `net_bench_two_region_env`: 800 KB/s -> 4 MB/s * `pfn_performance_with_realistic_env`: 4K -> 6K TPS
Description
Tests show that overriding the socket options for send and recv buffer actually hurts performance (even with default linux kernel settings). Logically, this rolls back #4649 but keeping the configs and code around in case this or other socket options are revisited.
Test Plan
New results:
net_bench_two_region_env
: 800 KB/s -> 4 MB/spfn_performance_with_realistic_env
: 4K -> 6K TPS