Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API Aggregator #4173

Merged
merged 8 commits into from
Jun 27, 2017
Merged

API Aggregator #4173

merged 8 commits into from
Jun 27, 2017

Conversation

chenopis
Copy link
Contributor

@chenopis chenopis commented Jun 22, 2017

kubernetes/features#263

/assign @cheftako


This change is Reviewable

@chenopis chenopis added this to the 1.7 milestone Jun 22, 2017
@chenopis chenopis self-assigned this Jun 22, 2017
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 22, 2017
@chenopis chenopis force-pushed the chenopis-api-aggregation branch 3 times, most recently from 825928b to b45c4f7 Compare June 23, 2017 00:09
@chenopis chenopis force-pushed the chenopis-api-aggregation branch from b45c4f7 to f9ab79a Compare June 23, 2017 00:10
@chenopis chenopis requested review from cheftako and lavalamp June 23, 2017 00:11
@chenopis
Copy link
Contributor Author

@lavalamp This is just the initial commit. I will fold in the changes from the Google doc once you guys have that ironed out.

@chenopis
Copy link
Contributor Author

@abiogenesis-now Can you help w/ a docs review? Thx

Copy link
Contributor

@jesscodez jesscodez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chenopis This is the first review I've done, so please take with a grain of salt! (And feel free to review my review). Github makes it a bit harder to suggest/comment on sentence structure than Google Docs 😞

@@ -20,6 +20,7 @@ toc:
section:
- docs/concepts/architecture/nodes.md
- docs/concepts/architecture/master-node-communication.md
- docs/concepts/architecture/apiserver-aggregation.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might make more sense to group this with/below the pre-existing Kubernetes API section.

You could either do this by bringing this section up to the Overview, or bringing the Kubernetes API section down underneath Kubernetes Architecture.

Feel free to disagree--I'm lacking context about dependencies or the history of the existing informational hierarchy, this is just my two cents. :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this should probably go along with the api section, or we could make a new api extension section and include this, TPR, CRD, & admission controllers / initializers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'm going to create a new subsection under Concepts called Extending the Kubernetes API and put this in there. We can then start adding other docs into that. I did originally try to put it in the Overview subsection next to Kubernetes API, but this doc isn't really an overview so it didn't quite fit.

{% capture whatsnext %}

* To get the aggregator working in your environment, see [Setup the API aggregator](/docs/tasks/access-kubernetes-api/setup-api-aggregator/).
* Learn how to [Extend the Kubernetes API Using Third Party Resources](/docs/tasks/access-kubernetes-api/extend-api-third-party-resource/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like TPRs are being deprecated in 1.7? Swap out with Extend the Kubernetes API Using Custom Resource Definitions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this might be the Google doc that you are already aware of, but it seems like TPR/CRDs are distinct from API Aggregation? It would be great to make this more clear, since this is a high-level conceptual doc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aggregator is completely distinct from TPR/CRD. However the Aggregator has no opinion on how the the extension resource data is persisted (or if it needs persisted data). Practically the extension resources will need to persist their data. The sample-apiserver and apiserver-builder both assume you will run your own etcd pod which they can use. TPR/CRD are also good answers to this need and allow kubernetes to solve the management portion of persisting the data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't call these third party resources. Extension APIs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the link has been changed


{% capture overview %}

The *Aggregator* is a system to allow the Kubernetes apiserver to be extended with additional APIs, which are not part of the core Kubernetes APIs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it best practice to refer to the "apiserver" or "API Server"? I've seen it both ways---it's more of a nit, but consistency would be nice.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the sentence reads a little easier as: "The Aggregator is a system that allows the Kubernetes apiserver to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs."?

Let me know if this comment is too fine-grained.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to refer to this as the "aggregation layer" since it is built in and not a separate component.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The aggregation layer allows Kubernetes to be extended with additional APIs which are not included in the kubernetes core APIs"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that makes a lot of sense.


{% capture whatsnext %}

* For more information, see the [Aggregator Networking Requirements](https://docs.google.com/document/d/1KNT4iS_Y2miLARrfSPumBIiFo_h7eb5B2pVOZJ0ZmjQ/edit) doc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have access to this Google doc---might want to check the permissions on it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to take this out until it's publicly accessible.


* For more information, see the [Aggregator Networking Requirements](https://docs.google.com/document/d/1KNT4iS_Y2miLARrfSPumBIiFo_h7eb5B2pVOZJ0ZmjQ/edit) doc.
* For a high level overview, see [Extending the Kubernetes API with Aggregator](/docs/concepts/architecture/apiserver-aggregation/).
* Learn how to [Extend the Kubernetes API Using Third Party Resources](/docs/tasks/access-Kubernetes-api/extend-api-third-party-resource/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see the earlier comment about CRDs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it


In 1.7, the Aggregator runs in a process with the kube-apiserver; however, until an extension resource is registered, the Aggregator will do nothing. Once a new resource is registered, Aggregator will proxy the request to the extension-apiserver to be served.

**Note:** The extension-apiserver will normally need to be paired with a controller to actually provide any functionality. As a result, things like the apiserver-builder will actually provide a skeleton for both. The service-catalog should provide both the extension-apiserver and controller for the services it provides.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ a bit wordy----am I correctly interpreting that both the apiserver-builder and the service-catalog provide the extension-apiserver and the controller? (A little confused if "should provide" means that service-catalog does it by default)

Copy link
Member

@cheftako cheftako Jun 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The apiserver-build will provide both a skeleton extension-apiserver and a skeleton controller. The service catalog can install extensions for you. When it does this it should provide/deploy both the extension apiserver and any other resources it needs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reworded this paragraph to clarify.


## Overview

The Aggregator allows both third party extension via something like [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md) and the creation of extensions to allow seamless addition of the resources a customer needs. The latter can be easily bootstrapped using [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not entirely clear to me what the difference between these two are, sorry! I'll read up on it later and see if I have suggestions, but do you mind clarifying?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Third party extensions are resource which are not built by either the cluster owner or the kubernetes team. Service Catalog is an example of this and is in fact an example which will make it easy for cluster owners to easily add other third party extensions to their cluster. This sort of extension is already built but does need to be deployed properly to the kubernetes cluster. If the cluster owner wants to create their own extension they first have to build something which adheres to the extension api-server requirements and then they have to work out how to deploy it correctly. So the story difference here is largely about the use case, though the technical details are very similar.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"aggregation layer"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The aggregation layer allows installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions (e.g., service catalog), or user-created APIs (apiserver-builder can get you started)."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this is much more clear!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed


* You need to have a Kubernetes cluster running.
* Make sure there are Aggregator certificates (both CA and proxy) setup on the Kubernetes apiserver.
* The proxy cert must have been signed by the relevant CA.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarify what "relevant" means?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubernetes (and the kube-apiserver) have multiple CAs. I want to make sure the proxy is signed by the aggregator CA and not say the master CA.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this should go together with the previous bullet point then?

## Get an extension api-server working with Aggregator

The generated skeleton, which comes with [apiserver-builder](https://github.com/Kubernetes-incubator/apiserver-builder/blob/master/README.md), should setup the skeleton server it creates to do all of these steps. You can also look at the sample-apiserver under Kubernetes for ways the following can be done.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

provide link to sample-apiserver? Also consider rewording the first sentence.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated skeleton, which comes with [apiserver-builder](https://github.com/Kubernetes-incubator/apiserver-builder/blob/master/README.md), should setup the skeleton server it creates to do all of these steps. You can also look at the sample-apiserver under Kubernetes for ways the following can be done.

1. Create the Kubernetes namespace you want to run your extension api-service in.
1. Create/get a CA cert to be used to sign the server cert the extension api-server uses for HTTPS.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do these easily map to code? sample commands might make it easier to follow.

@chenopis
Copy link
Contributor Author

@cheftako Can you suggest someone to do the tech review? Thx

@cheftako
Copy link
Member

cheftako commented Jun 23, 2017 via email

--proxy-client-cert-file=<path to aggregator proxy cert>
--proxy-client-key-file=<path to aggregator proxy key>

If you are not running kube-proxy on your master, as the [Kubernetes Architectural Roadmap](https://docs.google.com/a/google.com/document/d/1XkjVm4bOeiVkj-Xt1LgoGiqWsBfNozJ51dyI-ljzt1o/edit?usp=sharing) recommends, then you need to make sure that the system is enabled with the following apiserver flag. Again, this may have already been taken care of by your provider.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found this a tad confusing. Just to be clear the roadmap recommends not running kube-proxy on the master.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reworded it

@chenopis
Copy link
Contributor Author

@cheftako Can we also get a another set of eyes on this for a second tech review?

@cheftako
Copy link
Member

cheftako commented Jun 23, 2017 via email


{% endcapture %}

{% capture prerequisites %}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not prereqs, these are what this doc is going to tell you how to do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Are there any other prereqs then that they'll need before they can get started?


--enable-aggregator-routing=true

## Get an extension api-server working with Aggregator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this and everything below should go in a separate doc, "how to setup an extension API server".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'm splitting this into two docs.

@chenopis
Copy link
Contributor Author

@lavalamp Ok, thanks for the feedback. I'll address your feedback this weekend.

@chenopis chenopis force-pushed the chenopis-api-aggregation branch 2 times, most recently from 79a7e7f to e726565 Compare June 25, 2017 21:22
@chenopis chenopis force-pushed the chenopis-api-aggregation branch from e726565 to 34f2609 Compare June 25, 2017 21:24
@chenopis
Copy link
Contributor Author

I've addressed all the comments. @cheftako @lavalamp @abiogenesis-now PTAL.

Previews:
Concept/Extending the Kubernetes API with the aggregation layer
Task/Configure the aggregation layer
Task/Setup an extension API server

@chenopis
Copy link
Contributor Author

FYI, all feedback must be addressed and LGTMs given by EOD Tue, June 27th so that this can be merged for the 1.7 release on June 28th.

Copy link
Contributor

@jesscodez jesscodez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple more comments, once they are addressed I think that this is reasonable.

@chenopis I have personal thoughts on the lack of syntax/nomenclature/rendering standardization for Kubernetes Objects vs general concepts vs binaries 😝 but I'll save that for our naming meeting. (There might already be best practices that I am unaware of at this point in time).

{% capture prerequisites %}

* You need to have a Kubernetes cluster running.
* You must [configure the aggregation layer](/docs/tasks/configure-aggregation-layer) and enable the apiserver flags.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix URL (should be docs/tasks/access-kubernetes-api/configure-aggregation-layer)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch


{% capture overview %}

Configuring the aggregation layer allows the Kubernetes apiserver to be extended with additional APIs, which are not part of the core Kubernetes APIs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth linking "aggregation layer" back to docs/concepts/api-extension/apiserver-aggregation? There's a link at the bottom of the page too so I don't feel too strongly about this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea


If you use an existing 3rd party solution, such as [apiserver-builder](https://github.com/Kubernetes-incubator/apiserver-builder/blob/master/README.md), it should generate a skeleton and perform all of the following steps. Alternatively, you can also look at the [sample-apiserver](https://github.com/kubernetes/sample-apiserver/blob/master/README.md) under Kubernetes for ways the following can be done.

1. Make sure the APIService API is enabled (check --runtime-config); it is on by default, so it should be on unless it’s been deliberately turned off in your cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrap --runtime-config in backticks?

Also rephrase last part:

It should be on by default, unless it's been deliberately turned off in your cluster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


The aggregation layer enables installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions, such as [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md), or user-created APIs like [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), which can get you started.

In 1.7 the aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. Users can add an APIService object to register their API, which “claims” the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService. Ordinarily, the APIService will be implemented by an extension-apiserver in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. For example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a paragraph break before "Ordinarily"? Mostly for readability, but I think it counts as a reasonable topic shift.

Also might be nice to italicize extension-apiserver in that first sentence to denote that it's a technical term.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


In 1.7 the aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. Users can add an APIService object to register their API, which “claims” the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService. Ordinarily, the APIService will be implemented by an extension-apiserver in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. For example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.

**Note:** The extension-apiserver will need to be paired with a controller to provide any functionality. Fortunately, if you use [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), it will provide both a skeleton extension-apiserver and a skeleton controller. Alternatively, the [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md) can install extensions for you and should provide/deploy both the extension apiserver and any other resources it needs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove this entire part; it's covered by the previous paragraph.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by entire part I mean line 23 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

{% capture whatsnext %}

* [Setup an extension api-server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/) to work with the aggregation layer.
* For a high level overview, see [Extending the Kubernetes API with Aggregator](/docs/concepts/api-extension/apiserver-aggregation/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Aggregator/the aggregation layer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Sorry, was updating things out of order.

{% capture whatsnext %}

* If you haven't already, [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/) and enable the apiserver flags.
* For a high level overview, see [Extending the Kubernetes API with Aggregator](/docs/concepts/api-extension/apiserver-aggregation/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Aggregator/aggregation layer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duh. thanks!


The aggregation layer enables installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions, such as [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md), or user-created APIs like [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), which can get you started.

In 1.7 the aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. Users can add an APIService object to register their API, which “claims” the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService. Ordinarily, the APIService will be implemented by an extension-apiserver in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. For example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

want to rewrite the third sentence:

To register their API, users must add an APIService object, which "claims" the URL path in the Kubernetes API.

in order to better contrast with the second sentence (the aggregation layer doing nothing unless extension resource is registered)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to gut check my own understanding of things, so quick question:

Is it the case that kube-apiserver is an actual official binary, whereas extension-apiserver is a stand-in name that can refer to any roll-your-own custom API server?

(e.g. I should not expect to be able to literally run ./extension-apiserver unless I explicitly name the custom API server binary that way?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the aggregation layer proxying mentioned here something that kube-apiserver does or kube-proxy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cheftako @lavalamp Can you clarify?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3rd sentence swapped out


## Enable apiserver flags

Enable the aggregation layer via the following apiserver flags. They may have already been taken care of by your provider.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarification: Are these flags meant to be applied when I run kube-apiserver?

I'm pretty sure this is the case but I could be wrong. Either way, could we clarify that "apiserver" means kube-apiserver, given that there are now multiple API servers floating around?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cheftako @lavalamp Can you confirm that "apiserver" means kube-apiserver here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does mean kube-apiserver.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification.


## Setup an extension api-server to work with the aggregation layer

If you use an existing 3rd party solution, such as [apiserver-builder](https://github.com/Kubernetes-incubator/apiserver-builder/blob/master/README.md), it should generate a skeleton and perform all of the following steps. Alternatively, you can also look at the [sample-apiserver](https://github.com/kubernetes/sample-apiserver/blob/master/README.md) under Kubernetes for ways the following can be done.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit confused by this blurb at first (e.g. why the steps below lacked detail). Consider the following wording for more clarity?

The following steps describe how to set up an extension-apiserver at a high level. For a concrete example of how they can be implemented, you can look at the sample-apiserver in the Kubernetes repo.

Alternatively, you can use an existing 3rd party solution, such as apiserver-builder, which should generate a skeleton and automate all of the following steps for you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@chenopis
Copy link
Contributor Author

@abiogenesis-now I think I put in all your feedback. Let me know if I missed anything.

@chenopis
Copy link
Contributor Author

@cheftako @lavalamp PTAL

@jesscodez
Copy link
Contributor

LGTM---thanks for incorporating feedback!


In 1.7 the aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register their API, users must add an APIService object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService.

Ordinarily, the APIService will be implemented by an *extension-apiserver* in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. For example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As another example, when the service catalog

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

Copy link
Member

@cheftako cheftako left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - with the couple of notes I made.

@chenopis chenopis merged commit 025f2fa into release-1.7 Jun 27, 2017
@chenopis chenopis deleted the chenopis-api-aggregation branch June 27, 2017 07:23
dchen1107 pushed a commit that referenced this pull request Jun 30, 2017
* Minor fixes in the Deployment doc

Signed-off-by: Michail Kargakis <[email protected]>

* add NodeRestriction to admission-controllers (#3842)

* Admins Can Configure Zones in Storage Class

The PR #38505 (kubernetes/kubernetes#38505) added zones optional parameter to Storage Class for AWS and GCE provisioners.

That's why documentation needs to be updated accordingly.

* document custom resource definitions

* add host paths to psp (#3971)

* add host paths to psp

* add italics

* Update ConfigMap doc to explain TTL-based cache updates (#3989)

* Update ConfigMap doc to explain TTL-based cache updates

* swap word order

Change "When a ConfigMap being already consumed..." to "When a ConfigMap already being consumed..."

* Update NetworkPolicy docs for v1

* StorageOS Volume plugin

* Update GPU docs

* docs: HPA autoscaling/v2alpha1 status conditions

This commit documents the new status conditions feature for HPA
autoscaling/v2alpha1.  It demonstrates how to get the status conditions
using `kubectl describe`, and how to interpret them.

* Update description about NodeRestriction

kubelet node can alse create mirror pods for their own static pods.

* adding storage as a supported resource to node allocatable

Signed-off-by: Vishnu kannan <[email protected]>

* Add documentation for podpreset opt-out annotation

This adds the annotation for having the podpreset admission controller
to skip (opt-out) manipulating the pod spec.

Also, the annotation format for what presets have acted on a pod has
been modified to add a prefix of "podpreset-". The new naming makes it such
that there is no chance of collision with the newly introduced opt-out
annotation (or future ones yet to be added).

Opt-out annotation PR:
kubernetes/kubernetes#44965

* Update PDB documentation to explain new field (#3885)

* update-docs-pdb

* Addressed erictune@'s comments

* Fix title and add a TOC to the logging concept page

* Patch #4118 for typos

* Describe setting coredns server in nameserver resolv chain

* Address comments in PR #3997.

Comment is in
https://github.com/kubernetes/kubernetes.github.io/pull/3997/files/f6eb59c67e28efc298c87b1ef49a96bc6adacd1e#diff-7a14981f3dd8eb203f897ce6c11d9828

* Update task for DaemonSet history and rollback (#4098)

* Update task for DaemonSet history and rollback

Also remove mentions of templateGeneration field because it's deprecated

* Address comments

* removed lt and gt as operators (#4152)

* removed lt and gt as operators

* replace lt and gt for node-affinfity

* updated based on bsalamat review

* Initial draft of upgrade guide for kubeadm clusters.

In-place upgrades are supported between 1.6 and 1.7 releases. Rollback
instructions to come in a separate commit.

Fixes kubernetes/kubeadm#278

* Add local volume documentation (#4050)

* Add local volume documentation

* Add PV local volume example

* Patch PR #3999

* Add documentation for Stackdriver event exporter

* Add documentation about controller metrics

* Federation: Add task for setting up placement policies (#4075)

* Add task for setting up placement policies

* Update version of management sidecar in policy engine deployment

* Address @nikhiljindal's comments

- Lower case filenames
- Comments in policy
- Typo fixes
- Removed type LoadBalancer from OPA Service

* Add example that sets cluster selector

Per-@nikhiljindal's suggestion

* Fix wording and templating per @chenopis

* PodDisruptionBudget documentation Improvements (#4140)

* Changes from #3885

Title: Update PDB documentation to explain new field
Author: foxish

* Added Placeholder Disruptions Concept Guide

New file: docs/concepts/workloads/pods/disruptions.md
Intented contents: concept for Pod Disruption Budget,
 cross reference to Eviction and Preemption docs.
Linked from: concepts > workloads > pods

* Added placeholder Configuring PDB Task

New file: docs/tasks/run-application/configure-pdb.md
Intented contents: task for writing a Pod Disruption Budget.
Linked from: tasks > configuring-applications > configure pdb.

* Add refs to the "drain a node" task.

* Refactor PDB docs.

Move the "Requesting an eviction" section from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
-- which is going away -- to:
docs/tasks/administer-cluster/safely-drain-node.md

The move is verbatim, except for an introductory sentence.

Also added assignees.

* Refactor of PDB docs

Moved the section:
Specifying a PodDisruptionBudget
from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
to:
docs/tasks/run-application/configure-pdb.md
because that former file is going away.
Move is verbatim.

* Explain how Eviction tools should handle failures

* Refactor PDB docs

Move text from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
to:
docs/concepts/workloads/pods/disruptions.md

Delete the now empty:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md

Added a redirects_from section to the new doc, containing the path
of the now-deleted doc, plus all the redirects from the deleted
doc.

* Expand PDB Concept guide

Building on a little content from the old task,
greatly expanded the Disruptions concept
guide, including an abstract example.

* Update creating a pdb Task.

* Address review comments.

* Fixed for all cody-clark's review comments

* Address review comments from mml

* Address review comments from maisem

* Fix missing backtick

* Api and Kubectl reference docs updates for 1.7 (#4193)

* Fix includes groups

* Generated kubectl docs for 1.7

* Generated references docs for 1.7 api

* Document node authorization mode

* API Aggregator (#4173)

* API Aggregator

* Additional bullet points

* incorporated feedback for apiserver-aggregation.md

* split setup-api-aggregator.md into two docs and address feedback

* fix link

* addressed docs feedback

* incorporate feedback

* integrate feedback

* Add documentation for DNS stub domains (#4063)

* Add documentation for DNS stub domains

* add additional prereq

* fix image path

* review feedback

* minor grammar and style nits

* documentation for using hostAliases to manage hosts file (#4080)

* documentation for using hostAliases to manage hosts file

* add to table of contents

* review comments

* update the right command to see hosts file

* reformat doc based on suggestion and change some wording

* Fix typo for #4080

* Patch PR #4063

* Fix wording in placement policy task introduction

* Add update to statefulset concepts and basic tutorial (#4174)

* Add update to statefulset concpets and basic tutorial

* Address tech comments.

* Update ESIPP docs for new added API fields

* Custom resource docs

* update audit document with advanced audit features added in 1.7

* kubeadm v1.7 documentation updates (#4018)

* v1.7 updates for kubeadm

* Address review comments

* Address Luke's comments

* Encrypting secrets at rest and cluster security guide

* Edits for Custom DNS Documentation (#4207)

* reorganize custom dns doc

* format fixes

* Update version numbers to 1.7

* Patch PR #4140 (#4215)

* Patch PR #4140

* fix link and typos

* Update PR template

* Update TLS bootstrapping with 1.7 features

This includes documenting the new CSR approver built into the
controller manager and the kubelet alpha features for certificate
rotation.

Since the CSR approver changed over the 1.7 release cycle we need
to call out the migration steps for those using the alpha feature.
This document as a whole could probably use some updates, but the
main focus of this PR is just to get these features minimally
documented before the release.

* Federated ClusterSelector

formatting updates from review

* complete PR #4181 (#4223)

* complete PR #4181

* fix security link

* Extensible admission controller (#4092)

* extensible-admission-controllers

* Update extensible-admission-controllers.md

* more on initializers

* fixes

* Expand external admission webhooks documentation

* wrap at 80 chars

* more

* add reference

* Use correct apigroup for network policy

* Docs changes to PR #4092 (#4224)

* Docs changes to PR #4092

* address feedback

* add doc for --as-group in cli

Add doc for this pr:
kubernetes/kubernetes#43696
jesscodez pushed a commit that referenced this pull request Sep 22, 2017
* API Aggregator

* Additional bullet points

* incorporated feedback for apiserver-aggregation.md

* split setup-api-aggregator.md into two docs and address feedback

* fix link

* addressed docs feedback

* incorporate feedback

* integrate feedback
jesscodez pushed a commit that referenced this pull request Sep 22, 2017
* Minor fixes in the Deployment doc

Signed-off-by: Michail Kargakis <[email protected]>

* add NodeRestriction to admission-controllers (#3842)

* Admins Can Configure Zones in Storage Class

The PR #38505 (kubernetes/kubernetes#38505) added zones optional parameter to Storage Class for AWS and GCE provisioners.

That's why documentation needs to be updated accordingly.

* document custom resource definitions

* add host paths to psp (#3971)

* add host paths to psp

* add italics

* Update ConfigMap doc to explain TTL-based cache updates (#3989)

* Update ConfigMap doc to explain TTL-based cache updates

* swap word order

Change "When a ConfigMap being already consumed..." to "When a ConfigMap already being consumed..."

* Update NetworkPolicy docs for v1

* StorageOS Volume plugin

* Update GPU docs

* docs: HPA autoscaling/v2alpha1 status conditions

This commit documents the new status conditions feature for HPA
autoscaling/v2alpha1.  It demonstrates how to get the status conditions
using `kubectl describe`, and how to interpret them.

* Update description about NodeRestriction

kubelet node can alse create mirror pods for their own static pods.

* adding storage as a supported resource to node allocatable

Signed-off-by: Vishnu kannan <[email protected]>

* Add documentation for podpreset opt-out annotation

This adds the annotation for having the podpreset admission controller
to skip (opt-out) manipulating the pod spec.

Also, the annotation format for what presets have acted on a pod has
been modified to add a prefix of "podpreset-". The new naming makes it such
that there is no chance of collision with the newly introduced opt-out
annotation (or future ones yet to be added).

Opt-out annotation PR:
kubernetes/kubernetes#44965

* Update PDB documentation to explain new field (#3885)

* update-docs-pdb

* Addressed erictune@'s comments

* Fix title and add a TOC to the logging concept page

* Patch #4118 for typos

* Describe setting coredns server in nameserver resolv chain

* Address comments in PR #3997.

Comment is in
https://github.com/kubernetes/kubernetes.github.io/pull/3997/files/f6eb59c67e28efc298c87b1ef49a96bc6adacd1e#diff-7a14981f3dd8eb203f897ce6c11d9828

* Update task for DaemonSet history and rollback (#4098)

* Update task for DaemonSet history and rollback

Also remove mentions of templateGeneration field because it's deprecated

* Address comments

* removed lt and gt as operators (#4152)

* removed lt and gt as operators

* replace lt and gt for node-affinfity

* updated based on bsalamat review

* Initial draft of upgrade guide for kubeadm clusters.

In-place upgrades are supported between 1.6 and 1.7 releases. Rollback
instructions to come in a separate commit.

Fixes kubernetes/kubeadm#278

* Add local volume documentation (#4050)

* Add local volume documentation

* Add PV local volume example

* Patch PR #3999

* Add documentation for Stackdriver event exporter

* Add documentation about controller metrics

* Federation: Add task for setting up placement policies (#4075)

* Add task for setting up placement policies

* Update version of management sidecar in policy engine deployment

* Address @nikhiljindal's comments

- Lower case filenames
- Comments in policy
- Typo fixes
- Removed type LoadBalancer from OPA Service

* Add example that sets cluster selector

Per-@nikhiljindal's suggestion

* Fix wording and templating per @chenopis

* PodDisruptionBudget documentation Improvements (#4140)

* Changes from #3885

Title: Update PDB documentation to explain new field
Author: foxish

* Added Placeholder Disruptions Concept Guide

New file: docs/concepts/workloads/pods/disruptions.md
Intented contents: concept for Pod Disruption Budget,
 cross reference to Eviction and Preemption docs.
Linked from: concepts > workloads > pods

* Added placeholder Configuring PDB Task

New file: docs/tasks/run-application/configure-pdb.md
Intented contents: task for writing a Pod Disruption Budget.
Linked from: tasks > configuring-applications > configure pdb.

* Add refs to the "drain a node" task.

* Refactor PDB docs.

Move the "Requesting an eviction" section from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
-- which is going away -- to:
docs/tasks/administer-cluster/safely-drain-node.md

The move is verbatim, except for an introductory sentence.

Also added assignees.

* Refactor of PDB docs

Moved the section:
Specifying a PodDisruptionBudget
from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
to:
docs/tasks/run-application/configure-pdb.md
because that former file is going away.
Move is verbatim.

* Explain how Eviction tools should handle failures

* Refactor PDB docs

Move text from:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md
to:
docs/concepts/workloads/pods/disruptions.md

Delete the now empty:
docs/tasks/administer-cluster/configure-pod-disruption-budget.md

Added a redirects_from section to the new doc, containing the path
of the now-deleted doc, plus all the redirects from the deleted
doc.

* Expand PDB Concept guide

Building on a little content from the old task,
greatly expanded the Disruptions concept
guide, including an abstract example.

* Update creating a pdb Task.

* Address review comments.

* Fixed for all cody-clark's review comments

* Address review comments from mml

* Address review comments from maisem

* Fix missing backtick

* Api and Kubectl reference docs updates for 1.7 (#4193)

* Fix includes groups

* Generated kubectl docs for 1.7

* Generated references docs for 1.7 api

* Document node authorization mode

* API Aggregator (#4173)

* API Aggregator

* Additional bullet points

* incorporated feedback for apiserver-aggregation.md

* split setup-api-aggregator.md into two docs and address feedback

* fix link

* addressed docs feedback

* incorporate feedback

* integrate feedback

* Add documentation for DNS stub domains (#4063)

* Add documentation for DNS stub domains

* add additional prereq

* fix image path

* review feedback

* minor grammar and style nits

* documentation for using hostAliases to manage hosts file (#4080)

* documentation for using hostAliases to manage hosts file

* add to table of contents

* review comments

* update the right command to see hosts file

* reformat doc based on suggestion and change some wording

* Fix typo for #4080

* Patch PR #4063

* Fix wording in placement policy task introduction

* Add update to statefulset concepts and basic tutorial (#4174)

* Add update to statefulset concpets and basic tutorial

* Address tech comments.

* Update ESIPP docs for new added API fields

* Custom resource docs

* update audit document with advanced audit features added in 1.7

* kubeadm v1.7 documentation updates (#4018)

* v1.7 updates for kubeadm

* Address review comments

* Address Luke's comments

* Encrypting secrets at rest and cluster security guide

* Edits for Custom DNS Documentation (#4207)

* reorganize custom dns doc

* format fixes

* Update version numbers to 1.7

* Patch PR #4140 (#4215)

* Patch PR #4140

* fix link and typos

* Update PR template

* Update TLS bootstrapping with 1.7 features

This includes documenting the new CSR approver built into the
controller manager and the kubelet alpha features for certificate
rotation.

Since the CSR approver changed over the 1.7 release cycle we need
to call out the migration steps for those using the alpha feature.
This document as a whole could probably use some updates, but the
main focus of this PR is just to get these features minimally
documented before the release.

* Federated ClusterSelector

formatting updates from review

* complete PR #4181 (#4223)

* complete PR #4181

* fix security link

* Extensible admission controller (#4092)

* extensible-admission-controllers

* Update extensible-admission-controllers.md

* more on initializers

* fixes

* Expand external admission webhooks documentation

* wrap at 80 chars

* more

* add reference

* Use correct apigroup for network policy

* Docs changes to PR #4092 (#4224)

* Docs changes to PR #4092

* address feedback

* add doc for --as-group in cli

Add doc for this pr:
kubernetes/kubernetes#43696
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants