REST API blocking async: Solution A, block_in_place

[junderw]

I found something in electrs that could greatly increase the stability of the REST API response times. It's an extremely simple fix.

However, there are 2 ways to fix the problem, and they both have tradeoffs and it depends on the number of threads, the number of incoming requests per threads, etc. as to which of the solutions is better for performance.

Knowing how many threads prod uses, and how many requests / second on average it processes will change which solution is best.

If we have N threads waiting to accept requests, and at any given time there are less than N requests being processed, solution A is better.
If we have 5 x N or 10 x N requests waiting to be processed at any given time, solution B is better.
However, anything in between would need to A/B test to figure out which one is better.
There's also a possibility (if the number of requests being processed at any given time never reaches anywhere near N requests ( like 10% of N etc.) that this change won't increase performance a lot.
So I was wondering if at some point we could try an A/B/C test with the A: Solution A, B: Solution B, and C: No changes... and see if latency changes at all.

Out of curiosity, have you felt like the REST API's response time latency is unstable with a lot of variance?

[mononaut]

Tested ACK @ 94cfe27

[wiz]

[mononaut]

pasting in my earlier recommendations below - I marginally preferred solution A when I reviewed the two PRs, but if we can test real performance in production/staging that would be even better.

merge #58 first, then test #56 and #57 separately in production and compare performance.
otherwise just pick one: both are improvements, and the difference doesn't seem that large testing under heavy load (but I'd lean towards solution A, #56)