Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change RuleList Family to FAMILY_V4 to ensure conflict routes cleanup #119

Merged
merged 1 commit into from
Jul 1, 2020

Conversation

BSWANG
Copy link
Member

@BSWANG BSWANG commented Jul 1, 2020

kernel return incomplete rules on FAMILY_ALL.
Then container rule clean up will miss the conflict routes.
Cause system rule leak.

use FAMILY_V4 will return complete ip rule.
relate kernel commit: torvalds/linux@c454673

// ip rule count limit by cb->args[0]:
  list_for_each_entry_rcu(ops, &net->rules_ops, list) {
    if (idx < cb->args[0] || !try_module_get(ops->owner))
      goto skip;

    if (dump_rules(skb, cb, ops) < 0)
      break;

    cb->args[1] = 0;
skip:
    idx++;
  }

Signed-off-by: bingshen.wbs [email protected]

Then POD rule clean up will miss the conflict routes.
Cause system rule leak.

use FAMILY_V4 will return complete ip rule.
relate kernel commit: torvalds/linux@c454673

Signed-off-by: bingshen.wbs <[email protected]>
@BSWANG BSWANG merged commit 8f40263 into AliyunContainerService:master Jul 1, 2020
@BSWANG
Copy link
Member Author

BSWANG commented Jul 1, 2020

v1.0.10.204-g8f40263-aliyun

@BSWANG
Copy link
Member Author

BSWANG commented Nov 20, 2020

清理现有的,重启节点或者使用如下脚本清理:

# 声明现在存在的pod的对应关系
declare -A existIPs=()

# 添加现在存在的pod
for existIP in `ip route | grep cali | awk '{print $1}'`; do
    existIPs["$existIP"]="1"
done

# 添加现在存在的pod
for existIPInRule in `ip rule | grep '2048:' | grep -v '\[detached\]' | awk '{print $3}'`; do
    existIPs["$existIPInRule"]="1"
done

# 清理to规则
for ruleIP in `ip rule | grep '512:' | awk '{print $5}'`; do
    echo $ruleIP -- ${existIPs["$ruleIP"]}
    if [[ ${existIPs["$ruleIP"]} == "1" ]]; then
        echo "exists"
        existIPs["$ruleIP"]="0"
    else
        echo "deleting $ruleIP"
        ip rule del from all to $ruleIP lookup main pref 512
    fi
done

# 清理from规则
ip rule | grep '\[detached\]' | awk '{print $5}' | xargs -n1 ip rule delete iif

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant