Fix an issue where getReadyIngresses was not loading ready ingresses at startup

[norbjd]

Changes

• 🐛 fix getReadyIngresses not loading ingresses at startup

/kind bug

I've noticed that with latest kourier release (1.11.0), kourier was not loading at all ready ingresses at startup. This was confirmed by looking at the logs, after restarting the controller on a cluster with a few ready ingresses:

Priming the config with 0 ingresses

This log is generated by https://github.com/knative-sandbox/net-kourier/blob/knative-v1.11.0/pkg/reconciler/ingress/controller.go#L210. I would have expected to see Priming the config with 100 ingresses, as my cluster contained 100 ingresses.

After investigation, it turns out that the PR #1075 introduced a regression on getReadyIngresses method (https://github.com/knative-sandbox/net-kourier/blob/knative-v1.11.0/pkg/reconciler/ingress/controller.go#L313-L327):

ingresses, err := ingressLister.List(labels.SelectorFromSet(map[string]string{
        v1alpha1ingress.ClassAnnotationKey: config.KourierIngressClassName,
}))

This can't work, because we're passing an annotation (networking.knative.dev/ingress.class: kourier.ingress.networking.knative.dev), while List method is actually filtering using labels. As a result, this List operation will always return an empty slice.

So technically, #1075 solves the issue where kourier takes too long to start, but it's due to a bug: since there are always 0 ingress returned, there is no initial reconciliation; thus, the xds management server is started very quickly, and the readiness probe succeeds, marking kourier as ready pretty fast.

This PR aims to solves this bug. But, as I've rollbacked to the previous version of getReadyIngresses, I've "reintroduced" the slow startup issue (because it's loading all ingresses, as it should do I guess). This is why I've also copied here the changes done in #1066 (on kube api QPS and burst params).

With these 2 changes, kourier loads all ready ingresses at startup and is pretty quick to start.

I'm not sure if introducing KUBE_API_{BURST,QPM} on this PR is a good thing or not; but, I've done so because otherwise, I couldn't even restart kourier on a kind cluster with 500 ingresses (got errors, probably because I was rate limited by Kubernetes API).

Tests

With these changes, on a Kind cluster with 500 ingresses, Kourier took around 1s to restart and to be ready (after reloading the 500 ingresses, I can see the correct log Priming the config with 500 ingresses).

It's worth noting that increasing KUBE_API_{BURST,QPM} has a side effect on kourier in general: it allows to speed up the ingresses creation. I've tried creating 500 ingresses on a Kind cluster (in parallel, with a concurrency limit of 20), here are the results:

• on main branch: it took 8m18s to create the ingresses. Creating 1 ingress took around 20s, as I were creating 20 ingresses at a time in parallel
• with this PR (KUBE_API_{BURST,QPM}=200): it takes 20s (25x less) to create all 500 ingresses, with the same parallelism

Release Note

Fix an issue where kourier was not reloading ingresses at startup

[norbjd]

Kind ping to @nak3 (:wave:) as you've worked on #1075 and #1066.

[codecov]

Codecov Report

Merging #1079 (8af7a3f) into main (5e4fd61) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1079   +/-   ##
=======================================
  Coverage   80.95%   80.95%           
=======================================
  Files          18       18           
  Lines        1355     1355           
=======================================
  Hits         1097     1097           
  Misses        205      205           
  Partials       53       53           

[norbjd]

For reviewers: this is based on #1066, but I couldn't find any doc on these environment variables; is there a doc somewhere that I've missed? If so, I think it would be worth adding a small comment there.

[nak3]

Yes, we should add a small comment. Would you like to add a comment? Or I can send a follow-up PR.

[norbjd]

Done ✔️ 8af7a3f. Feel free to tell me if comment is not clear enough 👍

[nak3]

Thank you! Looks good to me.

[nak3]

/lgtm
/hold

Thank you so much @norbjd Please feel free to unhold if you will not include the comment about KUBE_API_ values as #1079 (comment)

[knative-prow]

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: norbjd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[nak3]

/cherry-pick release-1.11

[knative-prow-robot]

@nak3: once the present PR merges, I will cherry-pick it on top of release-1.11 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[nak3]

/lgtm
/hold cancel

[knative-prow-robot]

@nak3: new pull request created: #1081

In response to this:

/cherry-pick release-1.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.