Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Resource Documentation #4071

Merged
merged 1 commit into from
Jun 27, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions _data/concepts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ toc:

- title: Extending the Kubernetes API
section:
- docs/concepts/api-extension/custom-resources.md
- docs/concepts/api-extension/apiserver-aggregation.md

- title: Containers
Expand Down
1 change: 1 addition & 0 deletions _data/tasks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ toc:
- docs/tasks/access-kubernetes-api/http-proxy-access-api.md
- docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions.md
- docs/tasks/access-kubernetes-api/extend-api-third-party-resource.md
- docs/tasks/access-kubernetes-api/migrate-third-party-resource.md
- docs/tasks/access-kubernetes-api/configure-aggregation-layer.md
- docs/tasks/access-kubernetes-api/setup-extension-api-server.md

Expand Down
79 changes: 79 additions & 0 deletions docs/concepts/api-extension/custom-resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
title: Custom Resources
assignees:
- enisoc
- deads2k
---

{% capture overview %}
This page explains the concept of *custom resources*, which are extensions of the Kubernetes API.
{% endcapture %}

{% capture body %}
## Custom resources

A *resource* is an endpoint in the [Kubernetes API](/docs/reference/api-overview/) that stores a
collection of [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a
certain kind.
For example, the built-in *pods* resource contains a collection of Pod objects.

A *custom resource* is an extension of the Kubernetes API that is not necessarily available on every
Kubernetes cluster.
In other words, it represents a customization of a particular Kubernetes installation.

Custom resources can appear and disappear in a running cluster through dynamic registration,
and cluster admins can update custom resources independently of the cluster itself.
Once a custom resource is installed, users can create and access its objects with
[kubectl](/docs/user-guide/kubectl-overview/), just as they do for built-in resources like *pods*.

## Custom controllers

On their own, custom resources simply let you store and retrieve structured data.
It is only when combined with a *controller* that they become a true
[declarative API](/docs/concepts/overview/working-with-objects/kubernetes-objects/#understanding-kubernetes-objects).
The controller interprets the structured data as a record of the user's desired state,
and continually takes action to achieve and maintain that state.

A *custom controller* is a controller that users can deploy and update on a running cluster,
independently of the cluster's own lifecycle.
Custom controllers can work with any kind of resource, but they are especially effective when
combined with custom resources.
The [Operator](https://coreos.com/blog/introducing-operators.html) pattern is one example of such a
combination. It allows developers to encode domain knowledge for specific applications into an
extension of the Kubernetes API.

## CustomResourceDefinitions

[CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
(CRD) is a built-in API that offers a simple way to create custom resources.
Deploying a CRD into the cluster causes the Kubernetes API server to begin serving the specified
custom resource on your behalf.

This frees you from writing your own API server to handle the custom resource,
but the generic nature of the implementation means you have less flexibility than with
[API server aggregation](#api-server-aggregation).

CRD is the successor to the deprecated *ThirdPartyResource* (TPR) API, and is available as of
Kubernetes 1.7.

## API server aggregation

Usually, each resource in the Kubernetes API requires code that handles REST requests and manages
persistent storage of objects.
The main Kubernetes API server handles built-in resources like *pods* and *services*,
and can also handle custom resources in a generic way through [CustomResourceDefinitions](#customresourcedefinitions).

The [aggregation layer](/docs/concepts/api-extension/) allows you to provide specialized
implementations for your custom resources by writing and deploying your own standalone API server.
The main API server delegates requests to you for the custom resources that you handle,
making them available to all of its clients.

{% endcapture %}

{% capture whatsnext %}
* Learn how to [Extend the Kubernetes API with the aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/).
* Learn how to [Extend the Kubernetes API with CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
* Learn how to [Migrate a ThirdPartyResource to CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/).
{% endcapture %}

{% include templates/concept.md %}
7 changes: 4 additions & 3 deletions docs/concepts/overview/kubernetes-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,10 @@ Currently there are several API groups in use:
(e.g. `apiVersion: batch/v1`). Full list of supported API groups can be seen in [Kubernetes API reference](/docs/reference/).


There are two supported paths to extending the API.
1. [Third Party Resources](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/extending-api.md)
are for users with very basic CRUD needs.
There are two supported paths to extending the API with [custom resources](/docs/concepts/api-extension/custom-resources/):

1. [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
is for users with very basic CRUD needs.
1. Coming soon: users needing the full set of Kubernetes API semantics can implement their own apiserver
and use the [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/aggregated-api-servers.md)
to make it seamless for clients.
Expand Down
8 changes: 3 additions & 5 deletions docs/concepts/workloads/controllers/garbage-collection.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,11 +154,9 @@ kubectl delete replicaset my-repset --cascade=false
```

## Known issues
* In 1.6, garbage collection does not support non-core resources, e.g.,
resources added via ThirdPartyResource or via aggregated API servers. It will
support non-core resources in the future. When it does, garbage collector will
delete objects with ownerRefereneces referring to non-existent object of a
valid non-core resource.
* As of 1.7, garbage collection does not yet support
[custom resources](/docs/concepts/api-extension/custom-resources/),
such as those added through CustomResourceDefinition or aggregated API servers.

[Other known issues](https://github.com/kubernetes/kubernetes/issues/26120)

Expand Down
10 changes: 7 additions & 3 deletions docs/reference/api-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,13 @@ Currently, there are several API groups in use:
* The named groups are at REST path `/apis/$GROUP_NAME/$VERSION`, and use `apiVersion: $GROUP_NAME/$VERSION`
(for example, `apiVersion: batch/v1`). Full list of supported API groups can be seen in [Kubernetes API reference](/docs/reference/).

There is a supported path to extending the API:
* [Third Party Resources](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/extending-api.md)
are for users with very basic CRUD needs.
There are two supported paths to extending the API with [custom resources](/docs/concepts/api-extension/custom-resources/):

1. [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
is for users with very basic CRUD needs.
1. Coming soon: users needing the full set of Kubernetes API semantics can implement their own apiserver
and use the [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/aggregated-api-servers.md)
to make it seamless for clients.


## Enabling API groups
Expand Down
Original file line number Diff line number Diff line change
@@ -1,48 +1,36 @@
---
title: Extend the Kubernetes API with CustomResourceDefinitions
assignees:
- IanLewis
title: Extending the Kubernetes API Using Custom Resource Definitions
redirect_from:
- "/docs/user-guide/customresourcedefinitions/"
- "/docs/user-guide/customresourcedefinitions.html"
- "/docs/concepts/ecosystem/customresourcedefinitions/"
- "/docs/concepts/ecosystem/customresourcedefinitions.html"
- deads2k
- enisoc
---

* TOC
{:toc}
{% capture overview %}
This page shows how to install a [custom resource](/docs/concepts/api-extension/custom-resources/)
into the Kubernetes API by creating a CustomResourceDefinition.
{% endcapture %}

## What is a CustomResourceDefinition?
{% capture prerequisites %}
* Read about [custom resources](/docs/concepts/api-extension/custom-resources/).
* Make sure your Kubernetes cluster has a master version of 1.7.0 or higher.
{% endcapture %}

Kubernetes comes with many built-in API objects. However, there are often times
when you might need to extend Kubernetes with your own API objects in order to do custom automation.
{% capture steps %}
## Create a CustomResourceDefinition

`CustomResourceDefinition` objects are a way to extend the Kubernetes API with
a new API object type. The new API object type will be given an API endpoint
URL and support CRUD operations, and watch API. You can then create custom
objects using this API endpoint. You can think of `CustomResourceDefinitions`
as being much like the schema for a database table. Once you have created the
table, you can then start storing rows in the table. Once created,
`CustomResourceDefinitions` can act as the data model behind custom controllers
or automation programs.

A `CustomResourceDefinition` creates the REST API for a custom resource of your chosen name.

## Creating a CustomResourceDefinition

When you create a new `CustomResourceDefinition`, the Kubernetes API Server
reacts by creating a new RESTful resource path (namespaced or cluster-scoped)
depending on your request. As with existing built-in objects, deleting a
When you create a new *CustomResourceDefinition* (CRD), the Kubernetes API Server
reacts by creating a new RESTful resource path, either namespaced or cluster-scoped,
as specified in the CRD's `scope` field. As with existing built-in objects, deleting a
namespace deletes all custom objects in that namespace.
`CustomResourceDefinitions` themselves are non-namespaced and are available to all namespaces.
CustomResourceDefinitions themselves are non-namespaced and are available to all namespaces.

For example, if you save the following `CustomResourceDefinition` to `resourcedefinition.yaml`:
For example, if you save the following CustomResourceDefinition to `resourcedefinition.yaml`:

```yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
# name must be in the form: plural.group
# name must match the spec fields below, and be in the form: <plural>.<group>
name: crontabs.stable.example.com
spec:
# group name to use for REST API: /apis/<group>/<version>
Expand All @@ -54,9 +42,9 @@ spec:
names:
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
plural: crontabs
# singular name to be used as alias on the CLI and for display
# singular name to be used as an alias on the CLI and for display
singular: crontab
# kind is normally the CamelCased singular type. Your resource manifests use this
# kind is normally the CamelCased singular type. Your resource manifests use this.
kind: CronTab
# shortNames allow shorter string to match your resource on the CLI
shortNames:
Expand All @@ -66,27 +54,28 @@ spec:
And create it:

```shell
$ kubectl create -f resourcedefinition.yaml
customresourcedefinitions "crontabs.stable.example.com" created
kubectl create -f resourcedefinition.yaml
```

Then a new RESTful API endpoint is created at:
Then a new namespaced RESTful API endpoint is created at:

`/apis/stable.example.com/v1/namespaces/<namespace>/crontabs/...`
```
/apis/stable.example.com/v1/namespaces/*/crontabs/...
```

This endpoint URL can then be used to create and manage custom objects.
The `kind` of these objects will be `CronTab` from the spec of the
`CustomResourceDefinition` object we created above.
CustomResourceDefinition object you created above.


## Creating Custom Objects
## Create custom objects

After the `CustomResourceDefinition` object has been created you can create
After the CustomResourceDefinition object has been created, you can create
custom objects. Custom objects can contain custom fields. These fields can
contain arbitrary JSON.
In the following example, a `cronSpec` and `image` custom fields are set to the
contain arbitrary JSON.
In the following example, the `cronSpec` and `image` custom fields are set in a
custom object of kind `CronTab`. The kind `CronTab` comes from the spec of the
`CustomResourceDefinition` object we created above.
CustomResourceDefinition object you created above.

If you save the following YAML to `my-crontab.yaml`:

Expand All @@ -103,23 +92,36 @@ spec:
and create it:

```shell
$ kubectl create -f my-crontab.yaml
crontab "my-new-cron-object" created
kubectl create -f my-crontab.yaml
```

You can then manage our `CronTab` objects using kubectl. Note that resource
names are not case-sensitive when using kubectl:
You can then manage your CronTab objects using kubectl. For example:

```shell
$ kubectl get crontab
kubectl get crontab
```

Should print a list like this:

```console
NAME KIND
my-new-cron-object CronTab.v1.stable.example.com
```

You can also view the raw JSON data. Here you can see that it contains the custom `cronSpec` and `image` fields from the yaml you used to create it:
Note that resource names are not case-sensitive when using kubectl,
and you can use either the singular or plural forms defined in the CRD,
as well as any short names.

```yaml
$ kubectl get ct -o yaml
You can also view the raw JSON data:

```shell
kubectl get ct -o yaml
```

You should see that it contains the custom `cronSpec` and `image` fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would I also see cronSpec and image if I did kubectl describe crontabs my-new-cron-object?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if you use kubectl 1.7+. Are you suggesting to show that output instead? It looks like this:

Name:           my-new-cron-object
Namespace:      default
Labels:         <none>
Annotations:    <none>
API Version:    stable.example.com/v1
Kind:           CronTab
Metadata:
  Cluster Name: 
  Creation Timestamp:   2017-06-26T20:23:26Z
  Generation:           0
  Resource Version:     427
  Self Link:            /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
  UID:                  4f18b46a-5aad-11e7-9f5e-a0481c9805b1
Spec:
  Cron Spec:    * * * * /5
  Image:        old-image
Events:         <none>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, it's just that I tend to use the describe subcommand to get additional info about a K8s object (as opposed to get with the -o option). I don't feel super strongly on this though.

from the yaml you used to create it:

```console
apiVersion: v1
items:
- apiVersion: stable.example.com/v1
Expand All @@ -142,11 +144,17 @@ metadata:
resourceVersion: ""
selfLink: ""
```
{% endcapture %}

{% capture discussion %}
## Advanced topics

## Advanced Topics
### Finalizers
CustomResources (objects created in the schema defined by CustomResourceDefintions)
support finalizers. If you add a `metadata.finalizers` stanza like

*Finalizers* allow controllers to implement asynchronous pre-delete hooks.
Custom objects support finalizers just like built-in objects.

You can add a finalizer to a custom object like this:

```yaml
apiVersion: "stable.example.com/v1"
Expand All @@ -156,7 +164,21 @@ metadata:
- finalizer.stable.example.com
```

Then when the CustomResource is deleted, the `metadata.deletionTimestamp` will
be set and update watch events will be sent to a controller which can perform
finalization steps before removing the finalizer and deleting the object again.
This allows cleanup for CustomResources like "normal" Kubernetes APIs.
The first delete request on an object with finalizers merely sets a value for the
`metadata.deletionTimestamp` field instead of deleting it.
This triggers controllers watching the object to execute any finalizers they handle.

Each controller then removes its finalizer from the list and issues the delete request again.
This request only deletes the object if the list of finalizers is now empty,
meaning all finalizers are done.
{% endcapture %}

{% capture whatsnext %}
* Learn how to [Migrate a ThirdPartyResource to CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/).
{% endcapture %}

{% include templates/task.md %}




Loading