-
Notifications
You must be signed in to change notification settings - Fork 406
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
29 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
In edge scenario, edge node can be offline since edge network is unstable, | ||
thanks to node autonomy feature provided by yurt-hub, pods can still be working, | ||
however there are some cases we need to take care. | ||
|
||
### Flannel: keep Vtep mac address | ||
If we use flannel as CNI plugin, in VXLAN mode, a Vtep device (usually named as "flannel.1") is created on each host, | ||
and the VNI, VtepMAC info are recorded in kube node annotation, consumed by other nodes to be aware and apply | ||
the corresponding routes and forwarding rules. | ||
Each time node restarted, flannel will create Vtep device with new mac address, and update the info in node annotation. | ||
|
||
However, if node is offline, the update cannot be truly done. | ||
This will cause pod connectivity issues between edge nodes as other nodes are not aware of the mac address change. | ||
|
||
**Solution: Keep Vtep mac address in flannel** | ||
|
||
For each time flannel restarted and try to create new Vtep device, it reads the mac address from node annotation, | ||
either from kube-apiserver or yurt-hub local cache, and use it for Vtep device if exists. | ||
|
||
For this solution, there is minor code change in flannel. | ||
|
||
### IPAM: keep pod ip address | ||
In most cases, host-local IPAM is used for CNI plugin. It works by assigning free ip address for each new pod from node's CIDR range, | ||
and recording the assigned ips in local path, to denote them occupied. | ||
For some edge application, static pod ip is required, which means pod ip address cannot be changed when it's recreated. | ||
|
||
**Solution: Keep ip address in IPAM** | ||
|
||
To support this, we need to adjust the IPAM plugin to also record {ip}-{pod name} binding info, and pass pod name in request to IPAM. | ||
For incoming ip allocation request, IPAM use the binded ip if pod name already exists in local path. |