Support persistent connections (aka HTTP/1.1 Keep-Alive)

[clue]

No description provided.

[clue]

This depends on proper detection of message payloads first (empty bodies, chunked transfer encoding and content-length), see #96.

[clue]

Small status update: We've added support for proper request body detection via #104 a while ago, so it's about time we finally get back to this ticket! I've started working on this again and it looks like this only needs a rather trivial change, so expect a PR for this soon-ish

[bradjones1]

@clue Any update on what is still required on this? Would love to use this on a project I'm working on.

[clue]

@bradjones1 As much as I'd love to commit to specific date, I can only say that I will get to this feature as soon as time permits.

If you need this for a commercial project and want to sponsor this feature, feel free to reach out 👍

In the meantime, you can always use this project behind a reverse proxy such as Haproxy or nginx in which case the front server takes care of handling persistent connections. I hope this helps 👍

[bradjones1]

@clue Totally understand... I'm excited about the potential for this, though I've already had to do an alternative implementation on the project I was working on. I'll reach out next time for a quote.

[clue]

Small progress update: This should now be implemented for both the server side and the new client side (#368). I've started working on this a while back and actually have a working prototype. The implementation for both client and server side is actually quite simple.

For the client side: Once a complete response is received, put the now idle connection into a list of idle connections. If a new outgoing request is to be sent, check if one idle connection to this destination already exists. After a very short grace period (defaults to a millisecond or so, possibly configurable), close the connection if no new outgoing request is sent.

For the server side, this implementation looks somewhat similar. When sending a response, we keep the connection open and wait for another request over the same connection. If no request arrives, we should close the connection. Accordingly, we need access to the LoopInterface. We've resolved this last blocker via #361.

This way, consumers of this library do not have to take care of this feature. Also, by using a very short grace period, the loop does not keep running noticeably longer than without this feature. This doesn't matter for long running scripts, but it's important for short running scripts to not appear to be "blocked".

[mmoreram]

@clue is anyone working on this nowadays? I would like to help you with this development. Do you have any schedule for these small tasks?

[tlaverdure]

@clue is there an example of this implementation somewhere?

For the client side: Once a complete response is received, put the now idle connection into a list of idle connections. If a new outgoing request is to be sent, check if one idle connection to this destination already exists. After a very short grace period (defaults to a millisecond or so, possibly configurable), close the connection if no new outgoing request is sent.

[clue]

@tlaverdure Good catch. This ticket here was created before we've added the HTTP client implementation (#368), so this used to focus on the server side only (resolved via #405). I've just created #468 to keep track of HTTP keep-alive for the client side. Let's move the discussion over there, any contribution is welcome!