Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]change coreDNS config to support edge-autonomy #1037

Closed
lorrielau opened this issue Oct 27, 2022 · 4 comments
Closed

[BUG]change coreDNS config to support edge-autonomy #1037

lorrielau opened this issue Oct 27, 2022 · 4 comments
Labels
kind/bug kind/bug

Comments

@lorrielau
Copy link
Member

What happened:
In e2e-tests, after we disconnected cloud nodes and restarted coreDNS, coreDNS is not able to resolve addresses. It might be because current coreDNS version is too heavy for edge local cache.
That's why we skipped coreDNS edge-autonomy test in the new edge-autonomy implementation pr: # pr 1022
In addition, current coreDNS schedule strategy does not assure at least one coreDNS on edge, which makes edge autonomy failing for lack of dns resolution ability. Therefore, in e2e-test shell: run-e2e-tests.sh , we use kubectl patch to remove coreDNS deployment-template-tolerations and modify replicas to 1, to make sure coreDNS is scheduled to edge nodes, and there is one and only one coreDNS on edge. So that we can restart the only coreDNS.

What you expected to happen:

  1. Modify coreDNS config, so the right version of coreDNS(which is able to run in edge local cache) is used.
  2. Modify coreDNS schedule strategy, to make sure edge is loaded with coreDNS.

How to reproduce it (as minimally and precisely as possible):
Delete the ginkgo.Skip in test/e2e/autonomy.go, Line 184, and the coreDNS e2e-edge-autonomy-test spec will fail.

Anything else we need to know?:

Environment:

  • OpenYurt version: 1.0
  • Kubernetes version (use kubectl version): client: 1.19.0, server: 1.22.7
  • OS (e.g: cat /etc/os-release): Ubuntu 18.04.6 LTS
  • Kernel (e.g. uname -a): x86_64
  • Install tools:
  • Others:

others

/kind bug

@lorrielau
Copy link
Member Author

@Congrool Please see if you have anything to add on.

@lorrielau
Copy link
Member Author

The main problem is coreDNS does not work in edge-autonomy scenarios; previous issue 880 talked about this, but the problem is still there;It happened when:

  1. 'NODES_NUM=3 DISABLE_DEFAULT_CNI=true REGION=cn make docker-build-and-up-openyurt' to docker build-and-local-up a 3-nodes openyurt-cluster;
  2. Set up flannel, by 'kubectl apply -f flannel.yaml' which I got from flannel;
  3. Set up bridge cni, got from bridge cni ;
  4. Use 'kubectl patch' to delete coreDNS deploy tolerations, and scaled coreDNS replicas down to 1; to make sure there is one coreDNS on edge;
  5. Start edge-autonomy tests; in ginkgo.BeforeSuite, obtain the coreDNS service IP;
  6. Stop coreDNS on edge, use 'crictl stop' in the edge node;
  7. Periodically use 'dig' , specifying the dns server to be the coreDNS serviceIP
    the test results looks like this:

image

coreDNS failing the edge-autonomy test, indicates that coreDNS is not working after cloud-edge disconnection.

@rambohe-ch
Copy link
Member

@lorrielau Thank you for uploading the detail info of CoreDNS problem, and i will merge the e2e pull request first.

and i am looking forward to you will fix the problem of CoreDNS in another pull request

@rambohe-ch
Copy link
Member

@lorrielau dns autonomy in E2E tests is supported in #1045 , so i will close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug kind/bug
Projects
No open projects
Status: Done
Development

No branches or pull requests

2 participants