Skip to content

Latest commit

 

History

History
220 lines (181 loc) · 9.35 KB

yurtctl.md

File metadata and controls

220 lines (181 loc) · 9.35 KB
Raw

Yurtctl tutorial

This tutorial demonstrates how to use yurtctl to install/uninstall OpenYurt. Please refer to the Getting Started section on the README page to prepare and build binary to _output/bin/yurtctl . We assume a minikube cluster (version 1.14 or less) is installed.

Convert a minikube cluster

Let us use yurtctl to convert a standard Kubernetes cluster to an OpenYurt cluster.

  1. Run the following command
$ _output/bin/yurtctl convert --provider minikube
  1. yurtctl will install all required components and reset the kubelet in the edge node. The output looks like:
convert.go:148] mark minikube as the edge-node
convert.go:178] deploy the yurt controller manager
convert.go:190] deploying the yurt-hub and resetting the kubelet service...
util.go:137] servant job(yurtctl-servant-convert-minikube) has succeeded
  1. yurt controller manager and yurthub Pods will be up and running in one minute. Let us verify them:
$ kubectl get deploy yurt-controller-manager -n kube-system
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
yurt-ctrl-mgr   1/1     1            1           23h
$ kubectl get po yurt-hub-minikube -n kube-system
NAME                READY   STATUS    RESTARTS   AGE
yurt-hub-minikube   1/1     Running   0          23h
  1. Next, we mark desired edge nodes as autonomous (only pods running on the autonomous edge nodes will be prevented from being evicted during disconnection):
$ _output/bin/yurtctl markautonomous
I0602 14:11:08.610222   89160 markautonomous.go:149] mark minikube-m02 as autonomous
  1. As the minikube cluster only contains one node, the node will be marked as an autonomous edge node. Let us verify this by inspecting the node's labels and annotations:
$ kubectl describe node | grep Labels -A 5
Labels:      alibabacloud.com/is-edge-worker=true
$ kubectl describe node | grep Annotations -A 5
Annotations: node.beta.alibabacloud.com/autonomy: true

By now, the OpenYurt cluster is ready. Users will not notice any differences compared to native Kubernetes when operating the cluster. If you login to the node, you will find the local cache has been populated:

$ minikube ssh
$ ls /etc/kubernetes/cache/kubelet/
configmaps  events  leases  nodes  pods  secrets  services

Test node autonomy

To test if edge node autonomy works as expected, we will simulate a node "offline" scenario.

  1. Let's first create a sample pod:
kubectl apply -f-<<EOF
apiVersion: v1
kind: Pod
metadata:
  name: bbox
spec:
  containers:
  - image: busybox
    command:
    - top
    name: bbox
EOF
  1. Make the edge node "offline" by changing the yurthub's server-addr to an unreachable address:
$ minikube ssh
$ sudo sed -i 's|--server-addr=.*|--server-addr=https://1.1.1.1:1111|' /etc/kubernetes/manifests/yurt-hub.yaml
  1. Now yurthub is disconnected from the apiserver and works in offline mode. To verify this, we can do the following:
$ minikube ssh
$ curl -s http://127.0.0.1:10261
{
  "kind": "Status",
  "metadata": {

  },
  "status": "Failure",
  "message": "request( get : /) is not supported when cluster is unhealthy",
  "reason": "BadRequest",
  "code": 400
}
  1. After 40 seconds, the edge node status becomes NotReady, but the pod/bbox won't be evicted and keeps running on the node:
$ kubectl get node && kubectl get po
NAME       STATUS     ROLES    AGE   VERSION
minikube   NotReady   master   58m   v1.18.2
NAME   READY   STATUS    RESTARTS   AGE
bbox   1/1     Running   0          19m

Convert a multi-nodes Kubernetes cluster

An OpenYurt cluster may consist of some edge nodes and some nodes in the cloud site. yurtctl allows users to specify a list of cloud nodes that won't be converted.

  1. Start with a two-nodes ack cluster,
$ kubectl get node 
NAME                     STATUS   ROLES    AGE   VERSION
us-west-1.192.168.0.87   Ready    <none>   19h   v1.14.8-aliyun.1
us-west-1.192.168.0.88   Ready    <none>   19h   v1.14.8-aliyun.1
  1. You can convert only one node to edge node(i.e., minikube-m02) by using this command:
$ _output/bin/yurtctl convert --provider ack --cloud-nodes us-west-1.192.168.0.87
I0529 11:21:05.835781    9231 convert.go:145] mark us-west-1.192.168.0.87 as the cloud-node
I0529 11:21:05.861064    9231 convert.go:153] mark us-west-1.192.168.0.88 as the edge-node
I0529 11:21:05.951483    9231 convert.go:183] deploy the yurt controller manager
I0529 11:21:05.974443    9231 convert.go:195] deploying the yurt-hub and resetting the kubelet service...
I0529 11:21:26.075075    9231 util.go:147] servant job(yurtctl-servant-convert-us-west-1.192.168.0.88) has succeeded
  1. Node us-west-1.192.168.0.87 will be marked as a non-edge node. You can verify this by inspecting its labels:
$ kubectl describe node us-west-1.192.168.0.87 | grep Labels
Labels:             alibabacloud.com/is-edge-worker=false
  1. Same as before, we make desired edge nodes autonomous:
$ _output/bin/yurtctl markautonomous
I0602 11:22:05.610222   89160 markautonomous.go:149] mark us-west-1.192.168.0.88 as autonomous
  1. When the OpenYurt cluster contains cloud nodes, yurt controller manager will be deployed on the cloud node (in this case, the node us-west-1.192.168.0.87):
$ kubectl get pods -A -o=custom-columns='NAME:.metadata.name,NODE:.spec.nodeName'
NAME                                               NODE
yurt-controller-manager-6947f6f748-lxfdx           us-west-1.192.168.0.87

Setup Yurttunnel

Since version v0.2.0, users can setup the yurttunnel using yurtctl convert.

Assume that the origin cluster is a two-nodes minikube cluster:

$ kubectl get node -o wide
NAME           STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIME
minikube       Ready    master   72m   v1.18.3   172.17.0.3    <none>        Ubuntu 20.04 LTS   4.19.76-linuxkit   docker://19.3.8
minikube-m02   Ready    <none>   71m   v1.18.3   172.17.0.4    <none>        Ubuntu 20.04 LTS   4.19.76-linuxkit   docker://19.3.8

Then, by simply running the yurtctl convert command with the enabling of the option --deploy-yurttunnel, yurttunnel servers will be deployed on cloud nodes, and an yurttunnel agent will be deployed on every edge node.

$ yurtctl convert --deploy-yurttunnel --cloud-nodes minikube --provider minikube
I0831 12:35:51.719391   77322 convert.go:214] mark minikube as the cloud-node
I0831 12:35:51.728246   77322 convert.go:222] mark minikube-m02 as the edge-node
I0831 12:35:51.753830   77322 convert.go:251] the yurt-controller-manager is deployed
I0831 12:35:51.910440   77322 convert.go:270] yurt-tunnel-server is deployed
I0831 12:35:51.999384   77322 convert.go:278] yurt-tunnel-agent is deployed
I0831 12:35:51.999409   77322 convert.go:282] deploying the yurt-hub and resetting the kubelet service...
I0831 12:36:22.109338   77322 util.go:173] servant job(yurtctl-servant-convert-minikube-m02) has succeeded
I0831 12:36:22.109368   77322 convert.go:292] the yurt-hub is deployed

To verify that the yurttunnel works as expected, please refer to the yurttunnel tutorial

Revert/Uninstall OpenYurt

Using yurtctl to revert an OpenYurt cluster can be done by doing the following:

$ _output/bin/yurtctl revert
revert.go:100] label alibabacloud.com/is-edge-worker is removed
revert.go:110] yurt controller manager is removed
revert.go:124] ServiceAccount node-controller is created
util.go:137] servant job(yurtctl-servant-revert-minikube-m02) has succeeded
revert.go:133] yurt-hub is removed, kubelet service is reset

Note that before performing the uninstall, please make sure all edge nodes are reachable from the apiserver.

Troubleshooting

1. Failure due to pulling image timeout

The default timeout value of cluster conversion is 2 minutes. Sometimes pulling the related images might take more than 2 minutes. To avoid the conversion failure due to pulling images timeout, you can:

  • use mirrored image from aliyun container registry(ACR)
$ _output/bin/yurtctl convert --provider minikube \
  --yurt-controller-manager-image registry.cn-hangzhou.aliyuncs.com/openyurt/yurt-controller-manager:latest \
  --yurt-tunnel-agent-image registry.cn-hangzhou.aliyuncs.com/openyurt/yurt-tunnel-agent:latest \
  --yurt-tunnel-server-image registry.cn-hangzhou.aliyuncs.com/openyurt/yurt-tunnel-server:latest \
  --yurtctl-servant-image registry.cn-hangzhou.aliyuncs.com/openyurt/yurtctl-servant:latest \
  --yurthub-image registry.cn-hangzhou.aliyuncs.com/openyurt/yurthub:latest
  • or pull all images on the node manually or use automation tools such as broadcastjob(from Kruise) in advance.

2. Adhoc failure recovery

In case any adhoc failure makes the Kubelet fail to communicate with APIServer, one can recover the original Kubelet setup by running the following command in edge node directly:

$ sudo sed -i "s|--kubeconfig=.*kubelet.conf|--kubeconfig=/etc/kubernetes/kubelet.conf|g;" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf && sudo systemctl daemon-reload && sudo systemctl restart kubelet.service