Provide a way to limit the total duration of a client request

[jrhee17]

Background

Currently, client requests are limited by a series of cancellation schedulers.

An example could be as follows:

1. A request tries to initialize the endpoint
2. A request invokes a dns query
3. A connection is acquired for a request
4. A request writes to a connection
5. A request waits for the response

Each of the above steps are limited by a timeout (there is also an idle timeout which can be triggered between steps).

If a user would like to limit a request to X seconds, there is no easy way to do so beside tuning all relevant parameters.

Proposal

It has been proposed that it might be convenient for users if there was an ability to limit the total duration of a client request.

More specifically, it might be useful if users were able to specify the start time of the responseTimeout as an option.

There has been a couple of points from internal discussion:

• The totalTimeout should have the option to start from before decorator execution (so when ClientRequestContext is created)
  • If a timeout is invoked, ideally the decorator chain will halt execution.
• It might be possible to generalize if we give a ClientOption to specify the when to start the responseTimeout. (e.g. before dns, before connection, etc..)
  • We should take care to throw the appropriate exception depending on which stage the timeout has been invoked.
• Should we also generalize when the timeout ends?
  • The timing may be different between 1) when the response is read from the socket and parsed into a http response 2) when the returned http response is consumed by the client
  • It might be a good idea to cancel the cancellation scheduler when deserialization is complete (grpc, thrift, etc..).

Reference: https://line-armeria.slack.com/archives/C1NGPBUH2/p1672103603661809

[jrhee17]

More specifically, it might be useful if users were able to specify the start time of the responseTimeout as an option.

Let me try to add a POC for this implementation.

I guess the new API may look something like the following:

enum ResponseTimeoutStartTiming {
    CLIENT_EXECUTE,
    REQUEST_WRITE,
    RESPONSE_WRITE,
}

public AbstractClientOptionsBuilder startResponseTimeout(ResponseTimeoutStartTiming startTiming) {
    return this;
}

We may also add the analogous APIs to ClientFactory and Flags.