Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add discard cloud service filter in yurthub #440

Merged
merged 1 commit into from
Sep 7, 2021

Conversation

rambohe-ch
Copy link
Member

@rambohe-ch rambohe-ch commented Aug 31, 2021

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind bug
/kind documentation
/kind enhancement
/kind good-first-issue
/kind feature
/kind question
/kind design
/sig ai
/sig iot
/sig network
/sig storage
/sig storage

/kind feature

What this PR does / why we need it:

  • background:
    Kube-proxy(ipvs mode) will configure ipvs rules for LoadBalancer service on the edge node, so if pods(like yurt-tunnel-agent) on edge nodes use ingress ip of LoadBalancer service to access the cloud pods(like yurt-tunnel-server), the connection will be refused for ipvs rule will dnat the ingress ip to pod ip on the edge node.
    And end user have come across the same error that yurt-tunnel-agent failed to use ingress ip of x-tunnel-server-svc service to access yurt-tunnel-server. the detailed info is here: [BUG]Yurthub return tunnel-server-service endpoint to kube-proxy on edge node #447

  • solution:
    In order to make sure pods on edge nodes can use LoadBalancer service to access pods on cloud nodes, we need disable the kube-proxy dnat rule for LoadBalancer service. so we add a filter named discardCloudService for yurthub to discard LoadBalancer service for kube-proxy component.
    At the same time, some ClusterIP services(like kube-system/x-tunnel-server-internal-svc) are not need to aware by edge nodes, so we also discard these ClusterIP service in the new filter.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?


other Note

@openyurt-bot
Copy link
Collaborator

@rambohe-ch: GitHub didn't allow me to assign the following users: your_reviewer.

Note that only openyurtio members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind bug
/kind documentation
/kind enhancement
/kind good-first-issue
/kind feature
/kind question
/kind design
/sig ai
/sig iot
/sig network
/sig storage
/sig storage

/kind feature

What this PR does / why we need it:

  • background:
    Pods on edge nodes will fail to connect pods(like yurt-tunnel-server) on cloud nodes if LoadBalancer service is used. because traffic from edge nodes will dnat to backend pod ip on cloud node by kube-proxy, but traffic from edge nodes can not reach pods on cloud nodes directly.

  • solution:
    In order to make sure pods on edge nodes can use LoadBalancer service to access pods on cloud nodes, we need disable the kube-proxy dnat rule for LoadBalancer service. so we add a filter named discardCloudService for yurthub to discard LoadBalancer service for kube-proxy component.
    At the same time, some ClusterIP services(like kube-system/x-tunnel-server-internal-svc) are not need to aware by edge nodes, so we also discard these ClusterIP service in the new filter.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?


other Note

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openyurt-bot openyurt-bot added kind/feature kind/feature size/L size/L: 100-499 labels Aug 31, 2021
@openyurt-bot openyurt-bot added the approved approved label Aug 31, 2021
@rambohe-ch
Copy link
Member Author

/assign @Fei-Guo

@rambohe-ch
Copy link
Member Author

@Fei-Guo I have updated the context of this feature. and internalTrafficPolicy feature of Service in k8s v1.21 can not solve the above problem, so after v1.21 version, we need to keep discardcloudservice filter enabled.

@Fei-Guo
Copy link
Member

Fei-Guo commented Sep 7, 2021

/lgtm
/approve

@openyurt-bot openyurt-bot added the lgtm lgtm label Sep 7, 2021
@openyurt-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Fei-Guo, rambohe-ch

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openyurt-bot openyurt-bot merged commit 3315ccc into openyurtio:master Sep 7, 2021
@DrmagicE
Copy link
Member

DrmagicE commented Sep 9, 2021

@rambohe-ch Hi, I am working on "adding yurthub on cloud-side feature" recently. In order to enable yurt-tunnel DNS mode, the kube-proxy on the cloud-side needs the "kube-system/x-tunnel-server-internal-svc" endpoints, so maybe we shoud add an option to let the cloud-side components can still get the endpoint?

@rambohe-ch
Copy link
Member Author

@DrmagicE On cloud nodes, you can add --disabled-resource-filters=discardcloudservice parameter for yurthub to disable discardcloudservice filter.

MrGirl pushed a commit to MrGirl/openyurt that referenced this pull request Mar 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved approved kind/feature kind/feature lgtm lgtm size/L size/L: 100-499
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants