Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(network_routes): missing node ip -> error log #904

Merged
merged 1 commit into from
May 25, 2020

Conversation

aauren
Copy link
Collaborator

@aauren aauren commented May 20, 2020

@murali-reddy This goes towards fixing another rare, but outstanding bug with kube-router for the 1.0.0 release.

Before we used to raise an error when a node was missing an IP, but it turns out that this is not a required attribute of a node. And while it is rare that a node would be missing an IP address, a node doesn't require an IP address or a name or really much of anything in order to
exist.

This brings us to stronger conformance with the Kubernetes API and makes it so that kube-router logs errors rather than changing it's health status and potentially causing cascading failures across the fleet if a user adds a node like this.

Properly fixes #846

@murali-reddy
Copy link
Member

thanks @aauren

I think use of utils.GetNodeIP() in bgp_peers.go simply assumes node object has IP address and ignoring the errors. https://github.com/cloudnativelabs/kube-router/blob/v1.0.0-rc4/pkg/controllers/routing/bgp_peers.go#L48

checked all the references for utils.GetNodeIP(), rest of the calls look good.

@aauren
Copy link
Collaborator Author

aauren commented May 24, 2020

@murali-reddy Good catch!

I added the instance of GetNodeIP you pointed out from bgp_peers

@murali-reddy
Copy link
Member

Sorry. My bad. I should have been more clear (references outside bgp_peers.go are all clean). There are 2 more such usages left in bgp_peers.go. Both handler for add/delete node events ignore the error.

Before we used to raise an error when a node was missing an IP, but it
turns out that this is not a required attribute of a node. And while it
is rare that a node would be missing an IP address, a node doesn't
require an IP address or a name or really much of anything in order to
exist.

This brings us to stronger conformance with the Kubernetes API and makes
it so that kube-router logs errors rather than changing it's health
status and potentially causing cascading failures across the fleet if a
user adds a node like this.
@aauren
Copy link
Collaborator Author

aauren commented May 25, 2020

@murali-reddy I fixed the remaining references in bgp_peers. Take another look when you have time.

@murali-reddy murali-reddy merged commit cb48a7f into cloudnativelabs:master May 25, 2020
@murali-reddy
Copy link
Member

Thanks LGTM

mrueg pushed a commit to mrueg/kube-router that referenced this pull request May 31, 2020
Before we used to raise an error when a node was missing an IP, but it
turns out that this is not a required attribute of a node. And while it
is rare that a node would be missing an IP address, a node doesn't
require an IP address or a name or really much of anything in order to
exist.

This brings us to stronger conformance with the Kubernetes API and makes
it so that kube-router logs errors rather than changing it's health
status and potentially causing cascading failures across the fleet if a
user adds a node like this.
FabienZouaoui pushed a commit to SirDataFR/kube-router that referenced this pull request Jun 22, 2020
Before we used to raise an error when a node was missing an IP, but it
turns out that this is not a required attribute of a node. And while it
is rare that a node would be missing an IP address, a node doesn't
require an IP address or a name or really much of anything in order to
exist.

This brings us to stronger conformance with the Kubernetes API and makes
it so that kube-router logs errors rather than changing it's health
status and potentially causing cascading failures across the fleet if a
user adds a node like this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Uninitialized node causes the kube-router failure
2 participants