Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support read throttling by size #830

Closed
Smityz opened this issue Oct 19, 2021 · 0 comments · Fixed by #829
Closed

Support read throttling by size #830

Smityz opened this issue Oct 19, 2021 · 0 comments · Fixed by #829
Assignees
Labels
type/enhancement Indicates new feature requests
Milestone

Comments

@Smityz
Copy link
Contributor

Smityz commented Oct 19, 2021

Design

I use folly::BasicDynamicTokenBucket to implement a throttling controller, which can limit read throughput accurately.
the throttling controller is divided into two parts to count data:
token_bucket_throttling_controller::only_count is used to count the read throughput.
token_bucket_throttling_controller::control is used to return the error code for the new coming request if the read throughput exceeds our limit.
This design guarantees that the request which is processing now won't be interrupted.

Perf_counter

I reuse the _counter_recent_read_throttling_reject_count perf_counter to record the reject request.

Additional parameters

I add a parameter named read_throttling_by_size_request_count to control the default value for the token bucket in token_bucket_throttling_controller::control because we don't know how much throughput it will make at the beginning. In my test, the default value 1 is available.

How to use

like other read/write controllers, I save this value in the table-env. Due to not supporting delay method, you can use it like
table-env set replica.read_throttling_by_size 30M and it also supports the historical style like 20000*delay*100,20000*reject*100, but delay won't be used.

admin-cli example

Pegasus-AdminCli-1.1.0 » ls
+-----+-----------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
| ID  |   Name    | Partitions | Unhealthy | WriteUnhealthy | ReadUnhealthy | CreateTime | WReqRateLimit | WBytesRateLimit |
+-----+-----------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
| 309 | test      | 4          | 0         | 0              | 0             | 2021-11-01 |               |                 |
| 312 | tyz_test1 | 64         | 0         | 0              | 0             | 2021-11-01 |               |                 |
+-----+-----------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
Pegasus-AdminCli-1.1.0 » use tyz_test1
ok
Pegasus-AdminCli-1.1.0 » table-env set replica.read_throttling_by_size 100M
Pegasus-AdminCli-1.1.0 » table-env ls
{
  "replica.read_throttling_by_size": "100M"
}

Function test

image

Related PR

#829
XiaoMi/rdsn#938

Split PR

XiaoMi/rdsn#939 add an interface for rpc_holder to set error
XiaoMi/rdsn#940 add an interface to get read_throttling_reject_count perf counter
XiaoMi/rdsn#941 add a token_bucket_throttling_controller

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Indicates new feature requests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants