Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clarification for ResourceQuota concept #42685

Merged
merged 9 commits into from
Apr 22, 2024
Merged

clarification for ResourceQuota concept #42685

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e4d1341
clarification for ResourceQuota concept
pegasas Aug 23, 2023
8bd2ee5
Update content/en/docs/concepts/policy/resource-quotas.md
pegasas Aug 23, 2023
6a66166
Update content/en/docs/concepts/policy/resource-quotas.md
pegasas Aug 23, 2023
a25ed42
fix confusing expression
pegasas Aug 25, 2023
605f158
fix description
pegasas Apr 1, 2024
5e7dee8
specify built-in API and CRD
pegasas Apr 2, 2024
9e4ecf0
Update resource-quotas.md
pegasas Apr 2, 2024
d7cdaf6
Update content/en/docs/concepts/policy/resource-quotas.md
pegasas Apr 3, 2024
efed6fd
Update content/en/docs/concepts/policy/resource-quotas.md
pegasas Apr 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions content/en/docs/concepts/policy/resource-quotas.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,8 +146,8 @@ Refer to [Logging Architecture](/docs/concepts/cluster-administration/logging/)

## Object Count Quota

You can set quota for the total number of certain resources of all standard,
namespaced resource types using the following syntax:
You can set quota for *the total number of one particular resource kind* in the Kubernetes API,
using the following syntax:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My experiments showed that the <resource> in count/<resource> can be anything.
It can be something that doesn't exist at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So ... this doen't have to be a "standard" resource type, right?

Copy link
Contributor

@sftim sftim Oct 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although the API accepts any resource name in that field, this could mean:

  1. you can set any ResourceQuota, even for an API the cluster doesn't yet have
    • if the cluster already has that resource kind, you cannot add a new resource that would exceed the quota
    • if the cluster gains that resource kind, the quota starts to be enforced
  2. you can set any ResourceQuota, even for an API the cluster doesn't yet have
    • if the cluster already has that resource kind, you cannot add a new resource that would exceed the quota
    • if the cluster gains that resource kind, the quota starts to be enforced
    • however, all of the above only includes built-in APIs and CustomResourceDefinitions. An APIService backed by an aggregated API server can fail to enforce a ResourceQuota without being seen as defective.
  3. you can set any ResourceQuota, even for an API the cluster doesn't yet have
    • however, there is an internal list of API kinds where enforcement happens
    • if you set a ResourceQuota for an different resource kind, the ResourceQuota is not enforced, and the lack of enforcement is silent

Let's be clear on which of those two three cases applies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the 1st case is the correct answer, but has to double confirm this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's case 2 (very well considered). Though the "default" aggregated apiserver (APIService) enforces quota.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'standard' is indeed ambiguous, better to clearly list out: built-in API and CRD


* `count/<resource>.<group>` for resources from non-core groups
* `count/<resource>` for resources from the core group
Expand All @@ -165,16 +165,21 @@ Here is an example set of resources users may want to put under object count quo
* `count/jobs.batch`
* `count/cronjobs.batch`

The same syntax can be used for custom resources.
If you define a quota this way, it applies to Kubernetes' APIs that are part of the API server, and
to any custom resources backed by a CustomResourceDefinition. If you use [API aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) to
add additional, custom APIs that are not defined as CustomResourceDefinitions, the core Kubernetes
control plane does not enforce quota for the aggregated API. The extension API server is expected to
provide quota enforcement if that's appropriate for the custom API.
For example, to create a quota on a `widgets` custom resource in the `example.com` API group, use `count/widgets.example.com`.

When using `count/*` resource quota, an object is charged against the quota if it exists in server storage.
When using such a resource quota (nearly for all object kinds), an object is charged
against the quota if the object kind exists (is defined) in the control plane.
These types of quotas are useful to protect against exhaustion of storage resources. For example, you may
want to limit the number of Secrets in a server given their large size. Too many Secrets in a cluster can
actually prevent servers and controllers from starting. You can set a quota for Jobs to protect against
a poorly configured CronJob. CronJobs that create too many Jobs in a namespace can lead to a denial of service.

It is also possible to do generic object count quota on a limited set of resources.
There is another syntax only to set the same type of quota for certain resources.
The following types are supported:

| Resource Name | Description |
Expand All @@ -194,6 +199,8 @@ created in a single namespace that are not terminal. You might want to set a `po
quota on a namespace to avoid the case where a user creates many small pods and
exhausts the cluster's supply of Pod IPs.

You can find more examples on [Viewing and Setting Quotas](#viewing-and-setting-quotas).

## Quota Scopes

Each quota can have an associated set of `scopes`. A quota will only measure usage for a resource if it matches
Expand Down