Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A58 update: Prefer application_utilization to cpu_utilization #373

Merged
merged 2 commits into from
Jun 7, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions A58-client-side-weighted-round-robin-lb-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,20 +125,20 @@ In all languages, when a subchannel state is updated to `IDLE`, the LB policy im

### Subchannel Weights

To enforce `blackout_period` and `weight_expiration_period`, the LB policy tracks two timestamps along with the weight. `last_updated` tracks the last time a report with non-zero `cpu_utilization` and `qps` was received. `non_empty_since` tracks when the first non-zero load report was received, and it is reset when a subchannel comes back to `READY` or the weight expires so that it can be updated with the next non-zero load report. The blackout and expiration are enforced when the weight value is looked up.
To enforce `blackout_period` and `weight_expiration_period`, the LB policy tracks two timestamps along with the weight. `last_updated` tracks the last time a report with non-zero `application_utilization` and `qps` was received. `non_empty_since` tracks when the first non-zero load report was received, and it is reset when a subchannel comes back to `READY` or the weight expires so that it can be updated with the next non-zero load report. The blackout and expiration are enforced when the weight value is looked up.
markdroth marked this conversation as resolved.
Show resolved Hide resolved

The weight of a backend is calculated using `qps` (queries per second), `eps` (errors per second) and `cpu_utilization` reported by the backend. `eps` is only used when `error_utilization_penalty` is set to non-zero and applies a penalty based on the error rate calculated as `eps/qps`. The formula used is as follows:
The weight of a backend is calculated using `qps` (queries per second), `eps` (errors per second) and `application_utilization` reported by the backend. `application_utilization` is a metric defined by the server application. This value is expected to range between [0.0, 1.0] normally but may exceed 1.0 when the usage exceeds the reporter dependent soft limits. The policy defaults to `cpu_utilization` when `application_utilization` is not defined in the load report. `eps` is only used when `error_utilization_penalty` is set to non-zero and applies a penalty based on the error rate calculated as `eps/qps`. The formula used is as follows:
markdroth marked this conversation as resolved.
Show resolved Hide resolved

$$weight = \dfrac{qps}{cpu\\_utilization + \dfrac{eps}{qps} * error\\_utilization\\_penalty}$$
$$weight = \dfrac{qps}{application\\_utilization + \dfrac{eps}{qps} * error\\_utilization\\_penalty}$$
markdroth marked this conversation as resolved.
Show resolved Hide resolved

In Java and Go, the weight will be stored in a wrapped subchannel. However, in C++, that approach won't work, because we create a new subchannel object for each address whenever we get a new address list, and we don't want to lose existing weight information when that happens. So instead, we store the weights in a map keyed by address, and each subchannel list will take its own ref to the entry in the map for each subchannel in the list.

```
function UpdateWeight(qps, eps, cpu_utilization) {
if (cpu_utilization > 0 && qps > 0)
cpu_utilization += eps / qps * config.error_utilization_penalty;
function UpdateWeight(qps, eps, application_utilization) {
markdroth marked this conversation as resolved.
Show resolved Hide resolved
if (application_utilization > 0 && qps > 0)
application_utilization += eps / qps * config.error_utilization_penalty;
var new_weight =
cpu_utilization == 0 ? 0 : qps / cpu_utilization;
application_utilization == 0 ? 0 : qps / application_utilization;
if (new_weight == 0) return;
if (non_empty_since_ == infy) non_empty_since_ = now();
last_updated_ = now();
Expand Down