Ensure jsonrpc threading settings are sane #11267
Conversation
Starting with `jsonrpc` v14, the "server threads" setting is more important than before and the current default of 1 means the https server is effectively single-threaded. This PR proposes a new default of 4 (and ensures that crazy settings like e.g. `0` are bumped to at least `1`). Also included: some docs, tests and cosmetics.
dvdplm
added
A0-pleasereview 🤓
ordian
added
A8-looksgood 🦄
There was a problem hiding this comment.
All the changes look good, however the processing threads CLI is not used at all. It used to be used to create a separate thread pool, where RPC requests from all transports are dispatched after they are deserialized by the transport (implementation-wise it was a jsonrpc-core middleware). It seems it was lost during some refactorings.
I think we should consider bringing this back, while it doesn't have that much impact on HTTP server, it definitelly allows WS and IPC servers to process requests in parallel - currently it's going to be sequential.
Co-Authored-By: Tomasz Drwięga <[email protected]>
Co-Authored-By: Tomasz Drwięga <[email protected]>
parity/cli/mod.rs
Outdated
| @@ -506,18 +506,18 @@ usage! { | |||
| "--jsonrpc-hosts=[HOSTS]", | |||
| "List of allowed Host header values. This option will validate the Host header sent by the browser, it is additional security against some attack vectors. Special options: \"all\", \"none\",.", | |||
|
|
|||
| ARG arg_jsonrpc_threads: (usize) = 4usize, or |c: &Config| c.rpc.as_ref()?.processing_threads, | |||
| ARG arg_jsonrpc_threads: (Option<usize>) = Some(4), or |c: &Config| c.rpc.as_ref()?.processing_threads, | |||
There was a problem hiding this comment.
If the default is Some, should it be usize then? How a user can set it to None and does it make sense? Maybe we should keep an Option and use unwrap_or(4) instead?
There was a problem hiding this comment.
None here implies "use defaults", i.e. the user didn't set a value. Either way, for this particular line it's moot: the jsonrpc-threads option does nothing and I removed it in dc0d349 – essentially it was once used to set the size of a CpuPool but since #9657 it has not had any effect at all.
The equivalent code today sets up the tokio Executor which is used for almost all async subsystems (sync, price fetch, rpcs and more) so it is proooobably not something users should be fiddling with. If the need to do so arises we should either use separate executors or introduce some other mechanism of configuring and reserving threads for each subsystem.
dvdplm
added
A0-pleasereview 🤓
Co-Authored-By: Niklas Adolfsson <[email protected]>
…thub.com:paritytech/parity-ethereum into dp/fix/better-default-jsonrpc-threading-settings * 'dp/fix/better-default-jsonrpc-threading-settings' of github.com:paritytech/parity-ethereum: Update parity/deprecated.rs
niklasad1
added
the
A7-looksgoodtestsfail 🤖
dvdplm
added
B0-patch-stable 🕷
* master: Clarify what first_block `None` means (#11269) removed redundant VMType enum with one variant (#11266) Ensure jsonrpc threading settings are sane (#11267) Return Ok(None) when the registrar contract returns empty slice (#11257) Add a benchmark for snapshot::account::to_fat_rlps() (#11185) Fix misc compile warnings (#11258) simplify verification (#11249)
Starting with
jsonrpcv14, the "server threads" setting is more important than before and the current default of 1 means the https server is effectively single-threaded. This PR proposes a new default of 4 (and ensures that crazy settings like e.g.0are bumped to at least1).Also included: some docs, tests and cosmetics.
EDIT Thanks to @tomusdrw for pointing out that
processing_threadsactually has no effect. It was once used to set the size of aCpuPool, but since #9657 we use the tokio defaults:num_cpus::get()(== number of logical cores). We use the sameExecutorfor many subsytems and if we do want to make it configurable we should expose that setting differently (and/or switch to using multipleExecutors). In this PR I opted to deprecate--jsonrpc-threadsand remove the correspondingprocessing_threadsstruct member, because I think the defaults are fine.