Configurable rate limiting per client

[radekmie]

As @lennyburdette said in #1347 (comment), the current implementation of rate limiting is a global one, not a per client one. I think it's a good idea to have the latter in the Router as well.

1. Right now, if we'd like to implement it in a supergraph with N subgraphs, we'll have N implementations (potentially shared) to handle it. If it'd be possible to handle it in the Router, there's only one (or zero, if it'd be configurable and not a Rhai-script one).
2. A sliding window just like the one that is already there with some client differentiation would most likely cover almost all cases. AFAIK a client's IP and a header (or few) are the most common fingerprint.
3. Storing the information in memory is enough for us, but I can imagine it may be a problem for supergraphs with a massive number of users. In this case, it'd need external storage of some sort.
   • If it would rely on the same interface as cache (rewrite the caching API #1281), then it'd be easy to store it in the same Redis/Memcached/whatever instance (same like external cache for APQ #1401).

An idea of what the configuration could look like (heavily inspired by global_rate_limit):

traffic_shaping:
   router: # Rules applied to requests from clients to the router
     client_rate_limit: # 10 requests per 5 seconds per IP.
       capacity: 10
       interval: 5s
       fingerprint:
         - ip
   subgraphs: # Rules applied to requests from the router to individual subgraphs
     products:
       client_rate_limit: # 10 requests per 5 seconds per access key (header).
         capacity: 10
         interval: 5s
         fingerprint:
           - header: authorization
     users:
       client_rate_limit: # 10 requests per 5 seconds per (IP, access key) pair.
         capacity: 10
         interval: 5s
         fingerprint:
           - ip
           - header: authorization

[gwardwell]

My use case for Router-based rate-limiting would definitely benefit from a shared rate-limiting state. Currently, as new Router instances are spun up in a multi-instance deployment, clients would effectively have their rate limit scale up as well. Enforcing the rate limit across all instances of the Router seems like it would be an ideal way to mitigate that problem.