-
Notifications
You must be signed in to change notification settings - Fork 715
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot join node if addon-kubeproxy phase has not been run #1349
Comments
hi,
kube-proxy is an "essential" addon. |
@neolit123 Primarily using IPv6-only cluster with BGP/Calico routing where kube-proxy is, for now at least, not needed and I'm using Ingress instead of Services in general. There is no problem in installing it for me, except I would like not to have anything not used hanging around - I like simple :) Everything I need seems to work without kube-proxy in my case, so not "essential" in my world . Maybe there are other/better ways to do my setup nowadays, or more advanced features I'm not using that will maybe give me problems if using them, but I'm happy without for now. I also don't install CoreDNS phase using kubeadm, but deploys it manually afterwards because I wish to adjust something in the deployment-spec - I imagine someone would maybe like to do the same with kube-proxy too, where this would probably come up as a problem. What does kubeadm join do with the kube-proxy info/config it fetches? Are there anything that actually needs to be set up on the node? I imagine this is could be solved with something like if (configmap/kube-proxy exists) then do something; else, but I very might be wrong. |
thank you for the explanation. your use case seems quite plausible. we seem to have a bug, or rather something we didn't anticipate related to phases. when join is performed in 1.13 we now have a way to retrieve the init configuration from the control plane node to the worker node. as part of this process we also fetch the "component config" for kubelet and kube-proxy, yet if one of them is missing (e.g. kube-proxy phase omitted) we simply error out. ref: one possible solution would be to show warnings and not error out in such cases. |
Related issue: #1318 The underlying topic IMO is what are "the officially supported" cluster variants Vs custom variants you can achieve using/skipping phases. Considering the potential huge number of the custom variants, my personal opinion is that it is not possible for the Sig to provide support/guarantees about the join/upgrade workflows in the custom use cases. This is why I proposed following items in the v1.14 planning discussion
This is a possible workaround for the join workflow, but after that the users will be stuck during the upgrade workflow. I'm in favour of a more structured solution as explained above (and with a dedicated CI signal) |
we can discuss the list of join phases with @timothysc on Wednesday and then i can create a tracking issue for volunteers, so that someone works on this problem in parallel. does that sound good? |
Is there any chance that this is not a bug per se? More related to the question in this issue would be that network plugins seem to make use of the kube-proxy component to set up pod-to-pod communications and as such it is and should be kept as an essential addon (unless replace by the kube-router?). |
@alejandrox1 What do you mean by filtering rules? I can use NetworkPolicy with Calico just fine without having kube-proxy in the cluster. DNS server(s) is set to routable IP-address instead of a ServiceIP so the DNS-config could be either a list of static Pod-IP's, a Headless Service (in which case kube-proxy is not used) or could even be external. (Newest Calico releases also supports BGP-routed ServiceIP's - I don't know how it works under the hood yet, tho). Pod-to-Pod communication is also just routable IP's. The point is not relying on NAT which is what kube-proxy mainly does AFAIK but instead having a simple L3 network between pods. I understand that it's a lot of combinations with every phase possibly being left out, but would at least supporting the "add-ons" to be optional be an acceptable compromise? |
Could |
@pytimer This was discussed last kubeadm office hours
However this doesn't means that things cannot chang, but to make this happen help from the community is needed |
i will try to send a proposal patch for this in this cycle, but it will be in the lines of:
/assign |
TODO: document support level on these changes. |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
we decided to close my PR here as it's far from ideal: and punt this issue to the |
looks like we closed this with the idea that upgrade phases will solve it but this is also a problem in join as the title indicated: see this new similar ticket: |
Is this a BUG REPORT or FEATURE REQUEST?
Choose one: BUG REPORT
Versions
kubeadm version (use
kubeadm version
): 1.13.2kubeadm version: &version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-10T23:33:30Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"linux/amd64"}
Environment:
kubectl version
): 1.13.2uname -a
): 3.10.0What happened?
When bootstrapping a master using phases, leaving out kube-proxy addon, nodes are unable to join with kubeadm join.
Error:
unable to fetch the kubeadm-config ConfigMap: failed to get component configs: configmaps "kube-proxy" is forbidden: User "system:bootstrap:jgozlb" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
What you expected to happen?
Be able to join node without installing kube-proxy addon.
How to reproduce it (as minimally and precisely as possible)?
Anything else we need to know?
Running
kubeadm init phase addon kube-proxy
on the master and re-run the join command on the node again, join is successful.The text was updated successfully, but these errors were encountered: