generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 143
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add missing docs (external scheduler and plugin extender) (#287)
* add missing docs (external scheduler and plugin extender) * fix minor point * fix based on the suggestion * fix broken import * add the important note on the top
- Loading branch information
1 parent
9540f5d
commit cb2365e
Showing
16 changed files
with
509 additions
and
260 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| ## How to use your custom plugins in the simulator | ||
|
|
||
| This doc describes how to use your custom plugins in the scheduler running in the simulator. | ||
|
|
||
| ### 1. Add your custom plugin's registry in OutOfTreeRegistries function. | ||
|
|
||
| Please add your custom plugin's registry in `outOfTreeRegistries` in config package here: | ||
|
|
||
| [kube-scheduler-simulator/simulator/scheduler/config/plugin.go](/simulator/scheduler/config/plugin.go) | ||
|
|
||
| ### 2. Configure the scheduler to enable your custom plugin | ||
|
|
||
| You can configure the scheduler to use your custom plugins through KubeSchedulerConfiguration. | ||
|
|
||
| [Scheduler Configuration | Kubernetes](https://kubernetes.io/docs/reference/scheduling/config/) | ||
|
|
||
| You can change the scheduler configuration in Web UI or | ||
| by passing a default KubeSchedulerConfiguration file via the environment variable `KUBE_SCHEDULER_CONFIG_PATH`. | ||
|
|
||
| ### Example | ||
|
|
||
| We will explain the case where you want to add [nodenumber](../sample/nodenumber/plugin.go) plugin as example. | ||
|
|
||
| The nodenumber plugin is an example plugin that favors nodes that have the number suffix which is the same as the number suffix of the pod name. | ||
| And we can configure it via `NodeNumberArgs`. | ||
|
|
||
| First, you need to add registry for the nodenumber plugin to `outOfTreeRegistries`. | ||
|
|
||
| ```go | ||
| outOfTreeRegistries = runtime.Registry{ | ||
| // TODO(user): add your plugins registries here. | ||
| nodenumber.Name: nodenumber.New, | ||
| } | ||
| ``` | ||
|
|
||
| Now you can use the nodenumber plugin in the simulator. | ||
|
|
||
| If you apply this configuration to the scheduler, you can see the nodenumber plugin is working (and NodeNumberArgs is applied to the nodenumber plugin) in the simulator, | ||
| and see the nodenumber plugin's result like other in-tree plugins. | ||
|
|
||
| ```yaml | ||
| kind: KubeSchedulerConfiguration | ||
| apiVersion: kubescheduler.config.k8s.io/v1 | ||
| # .... | ||
| profiles: | ||
| - schedulerName: default-scheduler | ||
| plugins: | ||
| # .... | ||
| multiPoint: | ||
| enabled: | ||
| # .... | ||
| - name: NodeNumber # added | ||
| weight: 10 | ||
| ``` | ||
|  |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| ## External scheduler | ||
|
|
||
| This document describes how to use the external scheduler instead of the scheduler running in the simulator. | ||
|
|
||
| We use the [`externalscheduler` package](../pkg/externalscheduler); | ||
| the scheduler built with the [`externalscheduler` package](../pkg/externalscheduler) will export the scheduling results on each Pod annotation. | ||
|
|
||
| ### Use cases | ||
|
|
||
| - Running your scheduler instead of the default one in the simulator | ||
| - You can still see the scheduling results in web UI as well! | ||
| - Running your scheduler with the simulator feature in your cluster | ||
| - All Pods, scheduled by this scheduler, will get the scheduler results on its annotation while each scheduling is done as usual. | ||
| - Note that it has performance overhead in each scheduling cycle | ||
| since the scheduler needs to make additional effort to export the scheduling results. | ||
|
|
||
| ### Change your scheduler | ||
|
|
||
| Here, we assume you're registering your custom plugins in your scheduler like this: | ||
|
|
||
| ```go | ||
| // your scheduler's main package | ||
| func main() { | ||
| command := app.NewSchedulerCommand( | ||
| app.WithPlugin(yourcustomplugin.Name, yourcustomplugin.New), | ||
| ) | ||
|
|
||
| code := cli.Run(command) | ||
| os.Exit(code) | ||
| } | ||
| ``` | ||
|
|
||
| Then, you need to replace few lines to use the [`externalscheduler` package](../pkg/externalscheduler). | ||
|
|
||
| ```go | ||
| func main() { | ||
| command, cancelFn, err := externalscheduler.NewSchedulerCommand( | ||
| externalscheduler.WithPlugin(yourcustomplugin.Name, yourcustomplugin.New), | ||
| externalscheduler.WithPluginExtenders(noderesources.Name, extender.New), // [optional] see plugin-extender.md about PluginExtender. | ||
| ) | ||
| if err != nil { | ||
| klog.Info(fmt.Sprintf("failed to build the scheduler command: %+v", err)) | ||
| os.Exit(1) | ||
| } | ||
| code := cli.Run(command) | ||
| cancelFn() | ||
| os.Exit(code) | ||
| } | ||
| ``` | ||
|
|
||
| As you see, `externalscheduler.NewSchedulerCommand` has much similar interface as the `app.NewSchedulerCommand`. | ||
| You can register your plugins by `externalscheduler.WithPlugin` option. | ||
|
|
||
| Via this step, all Pods scheduled by this scheduler will get the scheduling results in the annotation like in the simulator! | ||
|
|
||
| ### Connect your scheduler to the kube-apiserver in the simulator | ||
|
|
||
| If you are here to run the scheduler built with [`externalscheduler` package](../pkg/externalscheduler) in your cluster, | ||
| you don't need to follow this step. | ||
|
|
||
| Let's connect your scheduler into the simulator. | ||
|
|
||
| First, you need to set `externalSchedulerEnabled: true` on [the simulator config](../config.yaml) | ||
| so that the scheduler in the simulator won't get started. | ||
|
|
||
| Next, you need to connect your scheduler into the simulator's kube-apiserver via KubeSchedulerConfig: | ||
|
|
||
| ```yaml | ||
| kind: KubeSchedulerConfiguration | ||
| apiVersion: kubescheduler.config.k8s.io/v1 | ||
| clientConnection: | ||
| kubeconfig: ./path/to/kubeconfig.yaml | ||
| ``` | ||
| You can use this [kubeconfig.yaml](./kubeconfig.yaml) to communicate with the simulator's kube-apiserver. | ||
| ### The example external scheduler | ||
| We have the sample external scheduler implementation in [./sample/external-scheduler](./sample/external-scheduler). | ||
| prerequisite: | ||
| 1. set `externalSchedulerEnabled: true` on [the simulator config](../config.yaml) | ||
| 2. run the simulator | ||
|
|
||
| ```shell | ||
| cd sample/external-scheduler | ||
| go run main.go --config scheduler.yaml | ||
| ``` | ||
|
|
||
| You'll see the simulator is working with the external scheduler. |
Oops, something went wrong.