Dynamically allocate and shrink server connections

[krizhanovsky]

The issue was originally described in #76 (comment) . The issue is crucial due to recent #660 : if there are a lot of non-idempotent requests or send queues are full, then we have limited throughput to upstream servers. Thus static number of connections can be wrong and it'd be good to establish a new server connection if current connections are blocked by non-idempotent requests. Meantime, if APM shows that the server is overloaded, then it could has sense to reduce number of server connections. Probably unused connections also can be shrunk - it could improve performance of schedulers. However, too frequent changing of connections number is also not wished due to relatively expensive schedulers updates.

Some dependence on #712: we need deeper study on the machine learning APM algorithms.

[krizhanovsky]

Crucial note from #498:

Long polling is another question that bothers me. Imagine we have 32k connections to backend servers, and have 32k clients that use long polling In this case server connections are depleted and we can't serve new client connections.