Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: local rate limit #2258

Merged
merged 21 commits into from
Dec 21, 2023
Merged

feat: local rate limit #2258

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
a922a61
local rate limit
zhaohuabing Nov 30, 2023
49dd741
e2e test
zhaohuabing Dec 1, 2023
cebb999
add default bucket for local rate limit
zhaohuabing Dec 2, 2023
f11d519
Merge branch 'main' into local-rate-limit
zhaohuabing Dec 8, 2023
c564c8d
set always_consume_default_token_bucket false
zhaohuabing Dec 12, 2023
a0385c3
set always_consume_default_token_bucket false
zhaohuabing Dec 12, 2023
4bec4cc
Merge branch 'main' into local-rate-limit
zhaohuabing Dec 15, 2023
ea42881
Merge branch 'main' into local-rate-limit
zhaohuabing Dec 20, 2023
0bb9b8e
address comments
zhaohuabing Dec 20, 2023
5c7ad9a
Merge branch 'main' into local-rate-limit
zhaohuabing Dec 20, 2023
b532711
fix e2e test
zhaohuabing Dec 20, 2023
37ce393
fix lint
zhaohuabing Dec 20, 2023
a8da2c7
fix e2e test
zhaohuabing Dec 20, 2023
7809964
fix e2e test
zhaohuabing Dec 20, 2023
e24f056
add comments to explain global rate limit rules
zhaohuabing Dec 20, 2023
bac22c9
fix gen
zhaohuabing Dec 20, 2023
07c8397
add comments to explain local rate limit rules
zhaohuabing Dec 20, 2023
4e229ba
address comments
zhaohuabing Dec 21, 2023
5ac3836
Merge branch 'main' into local-rate-limit
zhaohuabing Dec 21, 2023
c0bc74a
Merge branch 'main' into local-rate-limit
zhaohuabing Dec 21, 2023
b1cdeb1
fix gen
zhaohuabing Dec 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 60 additions & 12 deletions api/v1alpha1/ratelimit_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,35 +9,58 @@ package v1alpha1
// +union
type RateLimitSpec struct {
// Type decides the scope for the RateLimits.
// Valid RateLimitType values are "Global".
// Valid RateLimitType values are "Global" or "Local".
//
// +unionDiscriminator
Type RateLimitType `json:"type"`
// Global defines global rate limit configuration.
//
// +optional
Global *GlobalRateLimit `json:"global,omitempty"`

// Local defines local rate limit configuration.
//
// +optional
Local *LocalRateLimit `json:"local,omitempty"`
}

// RateLimitType specifies the types of RateLimiting.
// +kubebuilder:validation:Enum=Global
// +kubebuilder:validation:Enum=Global;Local
type RateLimitType string

const (
// GlobalRateLimitType allows the rate limits to be applied across all Envoy proxy instances.
// GlobalRateLimitType allows the rate limits to be applied across all Envoy
// proxy instances.
GlobalRateLimitType RateLimitType = "Global"

// LocalRateLimitType allows the rate limits to be applied on a per Envoy
// proxy instance basis.
LocalRateLimitType RateLimitType = "Local"
)

// GlobalRateLimit defines global rate limit configuration.
type GlobalRateLimit struct {
// Rules are a list of RateLimit selectors and limits.
// Each rule and its associated limit is applied
// in a mutually exclusive way i.e. if multiple
// rules get selected, each of their associated
// limits get applied, so a single traffic request
// might increase the rate limit counters for multiple
// rules if selected.
// Rules are a list of RateLimit selectors and limits. Each rule and its
// associated limit is applied in a mutually exclusive way. If a request
// matches multiple rules, each of their associated limits get applied, so a
// single request might increase the rate limit counters for multiple rules
// if selected. The rate limit service will return a logical OR of the individual
// rate limit decisions of all matching rules. For example, if a request
// matches two rules, one rate limited and one not, the final decision will be
// to rate limit the request.
//
// +kubebuilder:validation:MaxItems=16
Copy link
Member Author

@zhaohuabing zhaohuabing Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a few lines to explain what happens if a request matches multiple rules, originally from https://github.com/envoyproxy/ratelimit?tab=readme-ov-file#example-2

Rules []RateLimitRule `json:"rules"`
}

// LocalRateLimit defines local rate limit configuration.
type LocalRateLimit struct {
// Rules are a list of RateLimit selectors and limits. If a request matches
// multiple rules, the strictest limit is applied. For example, if a request
// matches two rules, one with 10rps and one with 20rps, the final limit will
// be based on the rule with 10rps.
Copy link
Member Author

@zhaohuabing zhaohuabing Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explaining how local rate limit behaves if a request matches multiple rules, originally from https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/local_rate_limit_filter#using-rate-limit-descriptors-for-local-rate-limiting

//
// +optional
// +kubebuilder:validation:MaxItems=16
Rules []RateLimitRule `json:"rules"`
}
Expand All @@ -49,8 +72,14 @@ type RateLimitRule struct {
// specific clients using attributes from the traffic flow.
// All individual select conditions must hold True for this rule
// and its limit to be applied.
// If this field is empty, it is equivalent to True, and
// the limit is applied.
//
// If no client selectors are specified, the rule applies to all traffic of
// the targeted Route.
//
// If the policy targets a Gateway, the rule applies to each Route of the Gateway.
// Please note that each Route has its own rate limit counters. For example,
// if a Gateway has two Routes, and the policy has a rule with limit 10rps,
zhaohuabing marked this conversation as resolved.
Show resolved Hide resolved
// each Route will have its own 10rps limit.
//
// +optional
// +kubebuilder:validation:MaxItems=8
Expand All @@ -70,6 +99,7 @@ type RateLimitRule struct {
type RateLimitSelectCondition struct {
// Headers is a list of request headers to match. Multiple header values are ANDed together,
// meaning, a request MUST match all the specified headers.
// At least one of headers or sourceCIDR condition must be specified.
//
// +listType=map
// +listMapKey=name
Expand All @@ -78,6 +108,7 @@ type RateLimitSelectCondition struct {
Headers []HeaderMatch `json:"headers,omitempty"`

// SourceCIDR is the client IP Address range to match on.
// At least one of headers or sourceCIDR condition must be specified.
//
// +optional
SourceCIDR *SourceMatch `json:"sourceCIDR,omitempty"`
Expand All @@ -91,6 +122,7 @@ const (
SourceMatchExact SourceMatchType = "Exact"
// SourceMatchDistinct Each IP Address within the specified Source IP CIDR is treated as a distinct client selector
// and uses a separate rate limit bucket/counter.
// Note: This is only supported for Global Rate Limits.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest creating separate fields for Local and Global or adding CEL validation we cannot support something like Distinct for local

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am inclined to add a CEL validation. It's simpler.

Copy link
Member Author

@zhaohuabing zhaohuabing Dec 20, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been validated when translating GatewayAPI to IR, and the Policy status is set to Invalid if the type is Distinct.

I tried to validate it with the below CEL rule but no luck:

// +kubebuilder:validation:XValidation:rule="self.rules.all(r, has(r.clientSelectors)? (r.clientSelectors.all(s, has(s.sourceCIDR) && has(s.sourceCIDR.type)? s.sourceCIDR.type!='Distinct' : true)) : true)", message="Distinct SourceMatchType is not supported for Local Rate Limit"

And verified it on https://playcel.undistro.io/.

However, k8s throws the below error, complaining this rule is invalid.

Error:      	Received unexpected error:
        	            	unable to install CRDs onto control plane: unable to create CRD instances: unable to create CRD "backendtrafficpolicies.gateway.envoyproxy.io": CustomResourceDefinition.apiextensions.k8s.io "backendtrafficpolicies.gateway.envoyproxy.io" is invalid: spec.validation.openAPIV3Schema.properties[spec].properties[rateLimit].properties[local].properties[rules].x-kubernetes-validations[0].rule: Invalid value: apiextensions.ValidationRule{Rule:"self.rules.all(r, has(r.clientSelectors)? (r.clientSelectors.all(s, has(s.sourceCIDR) && has(s.sourceCIDR.type)? s.sourceCIDR.type!='Distinct' : true)) : true)", Message:"Distinct SourceMatchType is not supported for Local Rate Limit", MessageExpression:""}: compilation failed: ERROR: <input>:1:5: type 'list_type:{elem_type:{message_type:"selfType61480208.@idx"}}' does not support field selection
        	            	 | self.rules.all(r, has(r.clientSelectors)? (r.clientSelectors.all(s, has(s.sourceCIDR) && has(s.sourceCIDR.type)? s.sourceCIDR.type!='Distinct' : true)) : true)

Maybe I can try CEL later in a follow-up PR.

SourceMatchDistinct SourceMatchType = "Distinct"
)

Expand Down Expand Up @@ -148,6 +180,7 @@ const (
// HeaderMatchDistinct matches any and all possible unique values encountered in the
// specified HTTP Header. Note that each unique value will receive its own rate limit
// bucket.
// Note: This is only supported for Global Rate Limits.
HeaderMatchDistinct HeaderMatchType = "Distinct"
)

Expand All @@ -162,3 +195,18 @@ type RateLimitValue struct {
//
// +kubebuilder:validation:Enum=Second;Minute;Hour;Day
type RateLimitUnit string

// RateLimitUnit constants.
const (
// RateLimitUnitSecond specifies the rate limit interval to be 1 second.
RateLimitUnitSecond RateLimitUnit = "Second"

// RateLimitUnitMinute specifies the rate limit interval to be 1 minute.
RateLimitUnitMinute RateLimitUnit = "Minute"

// RateLimitUnitHour specifies the rate limit interval to be 1 hour.
RateLimitUnitHour RateLimitUnit = "Hour"

// RateLimitUnitDay specifies the rate limit interval to be 1 day.
RateLimitUnitDay RateLimitUnit = "Day"
)
27 changes: 27 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

164 changes: 154 additions & 10 deletions charts/gateway-helm/crds/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,22 +211,32 @@ spec:
rules:
description: Rules are a list of RateLimit selectors and limits.
Each rule and its associated limit is applied in a mutually
exclusive way i.e. if multiple rules get selected, each
of their associated limits get applied, so a single traffic
request might increase the rate limit counters for multiple
rules if selected.
exclusive way. If a request matches multiple rules, each
of their associated limits get applied, so a single request
might increase the rate limit counters for multiple rules
if selected. The rate limit service will return a logical
OR of the individual rate limit decisions of all matching
rules. For example, if a request matches two rules, one
rate limited and one not, the final decision will be to
rate limit the request.
items:
description: RateLimitRule defines the semantics for matching
attributes from the incoming requests, and setting limits
for them.
properties:
clientSelectors:
description: ClientSelectors holds the list of select
description: "ClientSelectors holds the list of select
conditions to select specific clients using attributes
from the traffic flow. All individual select conditions
must hold True for this rule and its limit to be applied.
If this field is empty, it is equivalent to True,
and the limit is applied.
\n If no client selectors are specified, the rule
applies to all traffic of the targeted Route. \n If
the policy targets a Gateway, the rule applies to
each Route of the Gateway. Please note that each Route
has its own rate limit counters. For example, if a
Gateway has two Routes, and the policy has a rule
with limit 10rps, each Route will have its own 10rps
limit."
items:
description: RateLimitSelectCondition specifies the
attributes within the traffic flow that can be used
Expand All @@ -238,7 +248,8 @@ spec:
description: Headers is a list of request headers
to match. Multiple header values are ANDed together,
meaning, a request MUST match all the specified
headers.
headers. At least one of headers or sourceCIDR
condition must be specified.
items:
description: HeaderMatch defines the match attributes
within the HTTP Headers of the request.
Expand Down Expand Up @@ -276,7 +287,8 @@ spec:
x-kubernetes-list-type: map
sourceCIDR:
description: SourceCIDR is the client IP Address
range to match on.
range to match on. At least one of headers or
sourceCIDR condition must be specified.
properties:
type:
default: Exact
Expand Down Expand Up @@ -331,11 +343,143 @@ spec:
required:
- rules
type: object
local:
description: Local defines local rate limit configuration.
properties:
rules:
description: Rules are a list of RateLimit selectors and limits.
If a request matches multiple rules, the strictest limit
is applied. For example, if a request matches two rules,
one with 10rps and one with 20rps, the final limit will
be based on the rule with 10rps.
items:
description: RateLimitRule defines the semantics for matching
attributes from the incoming requests, and setting limits
for them.
properties:
clientSelectors:
description: "ClientSelectors holds the list of select
conditions to select specific clients using attributes
from the traffic flow. All individual select conditions
must hold True for this rule and its limit to be applied.
\n If no client selectors are specified, the rule
applies to all traffic of the targeted Route. \n If
the policy targets a Gateway, the rule applies to
each Route of the Gateway. Please note that each Route
has its own rate limit counters. For example, if a
Gateway has two Routes, and the policy has a rule
with limit 10rps, each Route will have its own 10rps
limit."
items:
description: RateLimitSelectCondition specifies the
attributes within the traffic flow that can be used
to select a subset of clients to be ratelimited.
All the individual conditions must hold True for
the overall condition to hold True.
properties:
headers:
description: Headers is a list of request headers
to match. Multiple header values are ANDed together,
meaning, a request MUST match all the specified
headers. At least one of headers or sourceCIDR
condition must be specified.
items:
description: HeaderMatch defines the match attributes
within the HTTP Headers of the request.
properties:
name:
description: Name of the HTTP header.
maxLength: 256
minLength: 1
type: string
type:
default: Exact
description: Type specifies how to match
against the value of the header.
enum:
- Exact
- RegularExpression
- Distinct
type: string
value:
description: Value within the HTTP header.
Due to the case-insensitivity of header
names, "foo" and "Foo" are considered
equivalent. Do not set this field when
Type="Distinct", implying matching on
any/all unique values within the header.
maxLength: 1024
type: string
required:
- name
type: object
maxItems: 16
type: array
x-kubernetes-list-map-keys:
- name
x-kubernetes-list-type: map
sourceCIDR:
description: SourceCIDR is the client IP Address
range to match on. At least one of headers or
sourceCIDR condition must be specified.
properties:
type:
default: Exact
type: string
value:
description: Value is the IP CIDR that represents
the range of Source IP Addresses of the
client. These could also be the intermediate
addresses through which the request has
flown through and is part of the `X-Forwarded-For`
header. For example, `192.168.0.1/32`, `192.168.0.0/24`,
`001:db8::/64`.
maxLength: 256
minLength: 1
type: string
required:
- value
type: object
type: object
maxItems: 8
type: array
limit:
description: Limit holds the rate limit values. This
limit is applied for traffic flows when the selectors
compute to True, causing the request to be counted
towards the limit. The limit is enforced and the request
is ratelimited, i.e. a response with 429 HTTP status
code is sent back to the client when the selected
requests have reached the limit.
properties:
requests:
type: integer
unit:
description: RateLimitUnit specifies the intervals
for setting rate limits. Valid RateLimitUnit values
are "Second", "Minute", "Hour", and "Day".
enum:
- Second
- Minute
- Hour
- Day
type: string
required:
- requests
- unit
type: object
required:
- limit
type: object
maxItems: 16
type: array
type: object
type:
description: Type decides the scope for the RateLimits. Valid
RateLimitType values are "Global".
RateLimitType values are "Global" or "Local".
enum:
- Global
- Local
type: string
required:
- type
Expand Down
Loading