Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CPLB virtual address fall back for kube API URL #772

Merged
merged 4 commits into from
Oct 22, 2024

Conversation

kke
Copy link
Contributor

@kke kke commented Oct 10, 2024

Fixes k0sproject/k0s#771
Fixes k0sproject/k0s#773

If the k0s config specifies control plane load balancing with keepalived and lists virtual addresses, use the first of those when generating an external URL for the cluster's kube API.

Before:

  1. spec.api.externalAddress
  2. leader address + spec.api.port (or 6443)
  3. leader private address + spec.api.port (or 6443)

After:

  1. spec.api.externalAddress
  2. spec.network.controlpanelLoadBalancing.virtualServers[0]
  3. leader address + spec.api.port (or 6443)
  4. leader private address + spec.api.port (or 6443)

This external address is used when modifying the admin kubeconfig received from a leader and when validating worker connectivity to the kube api.

Cluster interal kube API URL is generated from the effective leader's private address or the public one if one is not defined.

When validating the kube API to come up on a node, the new k0s spec.api.onlyBindToAddress option is now considered, so instead of https://localhost it will use the configured address when the setting is enabled as the node local api address.

@kke kke added bug Something isn't working enhancement New feature or request labels Oct 10, 2024
@kke kke force-pushed the validate-connection-vrrp-ip branch from 2f8ea92 to b29d590 Compare October 18, 2024 06:28
@kke kke marked this pull request as ready for review October 18, 2024 07:06
@kke kke changed the title Fall back to CPLB virtual address in external API URL CPLB virtual address fall back for kube API URL Oct 18, 2024
@@ -309,6 +309,10 @@ func (p *ConfigureK0s) configFor(h *cluster.Host) (string, error) {
cfg.DigMapping("spec", "api")["address"] = addr
addUnlessExist(&sans, addr)

if externalAddr := cfg.DigString("spec", "api", "externalAddress"); externalAddr != "" {
addUnlessExist(&sans, externalAddr)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should externalAddr be in sans?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's added automatically. the component endpoint-reconciler should be disabled though. Everything else seems fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think ctl needs to worry about those. Maybe all the other sans manipulation is pointless too if k0s does it on its own?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Sans return the given SANS plus all local addresses and externalAddress if given
func (a *APISpec) Sans() []string {
	sans, _ := iface.AllAddresses()
	sans = append(sans, a.Address)
	sans = append(sans, a.SANs...)
	if a.ExternalAddress != "" {
		sans = append(sans, a.ExternalAddress)
	}

	return stringslice.Unique(sans)
}

k0s does all this, I don't think ctl needs to manipulate sans at all.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ctl added the FQDNs to the SANs (see https://github.com/k0sproject/k0s/issues/5168).

@kke kke force-pushed the validate-connection-vrrp-ip branch from 65f0c0c to fd0ba50 Compare October 22, 2024 07:14
@kke kke merged commit 4c6450d into main Oct 22, 2024
39 checks passed
@kke kke deleted the validate-connection-vrrp-ip branch October 22, 2024 12:11
kke added a commit that referenced this pull request Nov 4, 2024
kke added a commit that referenced this pull request Nov 4, 2024
kke added a commit that referenced this pull request Nov 7, 2024
kke added a commit that referenced this pull request Nov 7, 2024
kke added a commit that referenced this pull request Nov 8, 2024
* Restore sans manipulation that was removed in #772

Signed-off-by: Kimmo Lehto <[email protected]>

* Make sans global

Signed-off-by: Kimmo Lehto <[email protected]>

---------

Signed-off-by: Kimmo Lehto <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
3 participants