HTTPS for Loadbalancer Health Monitor

[MPV]

/kind feature

Describe the solution you'd like

Be able to use HTTPS as type for the load balancer's health monitor (on the /readyz path).

Anything else you would like to add:

Hopefully this would solve problems such as:

• TCP for Loadbalancer Monitor causes API server downtime during upgrades #1749

Related issues:

1. Most-recent improvement (to avoid incidents seen in Configurable api server loadbalancer monitor #1221 and Hard-coded health monitors (downtime during upgrades) #1374):
   • 🐛 fix: improve load balancer health checks #1375
2. Making load balancer health checks configurable overall (though only TCP settings discussed yet):
   • Configurable api server loadbalancer monitor #1221
   • ✨ Add loadbalancer monitor configuration #1360

[MPV]

Here's for example how OpenShift documents how to configure the load balancer health checks in this manner (and why not to use TCP port probe, but instead the /readyz endpoint):

• https://github.com/openshift/installer/blob/master/docs/dev/kube-apiserver-health-check.md

[MPV]

Though on the topic of setting/syncing the monitor's values with the api server's --shutdown-delay-duration setting, while searching for shutdown-delay-duration in CAPI repo, I found issues like these:

• KubeadmControlPlane v2 iteration and robustness cluster-api#2753 (mentioned, but solved other issues)
• KCP should shut down kube-apiserver gracefully cluster-api#2937
  ...closed in favor of:
  • Load Balancer Provider cluster-api#1250
    ...closed in favor of:
    • 📖 CAEP: Flexible Managed Kubernetes Endpoints cluster-api#8500
      ...merged 2 days ago, aiming to solve:
      • Managed Kubernetes in CAPI cluster-api#7494

[MPV]

I think we should also take into account what @mnaser said over in #1374:

I would have liked to change the following too:

• type: HTTPS
• url_path: /healthz

This will give us actual health checks that involve the API endpoint telling us that it's happy, and it should be OK to accept the unknown cert since openstack/octavia@8c793f2/octavia/common/jinja/haproxy/split_listeners/templates/macros.j2#L178-L182 is OK with it however OVN doesnt support this so we can keep TCP for now.

[MPV]

For example, here's one current monitor:

And here's a manually drafted one which I'd like to be able to use instead:

[mdbooth]

@dulek What do you think about using HTTPS health monitors here? There's a question about how to support the OVN provider. Does it support HTTPS?

[dulek]

@dulek What do you think about using HTTPS health monitors here? There's a question about how to support the OVN provider. Does it support HTTPS?

It makes perfect sense from my perspective as long as it's opt-in. The OVN provider does not support HTTP or HTTPS health monitors and most likely never will as it lives in layers below. That shouldn't block us from trying to improve things for Amphora.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[dulek]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[EmilienM]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.