Add e2e tests for Metrics Server and Master Metrics API

[nikhiljindal]

Need e2e tests for kubernetes/enhancements#271 which is being released as part of release 1.8.

cc @piosz who is working on the tests.

[grodrigues3]

/cc @DirectXMan12

[luxas]

@nikhiljindal Is this blocking the v1.8 release?

[nikhiljindal]

Yes. I am hoping that we wont ship 1.8 without any e2e tests for this feature?

I will let @piosz and the release team take a call, if it comes to that. Hopefully we will have e2e tests soon without causing delays to the release.

[ericchiang]

This is extremely late to be adding an e2e tests considering we just spent the last three weeks stabilizing the existing tests :(

If e2e tests are absolutely required lets get them in ASAP.

Removing the failing tests labels because this isn't a failing test.

cc @kubernetes/kubernetes-release-managers

[k8s-github-robot]

[MILESTONENOTIFIER] Milestone Labels Complete

@nikhiljindal @piosz

Issue label settings:

• sig/instrumentation: Issue will be escalated to these SIGs if needed.
• priority/critical-urgent: Never automatically move out of a release milestone; continually escalate to contributor and SIG through all available channels.
• kind/cleanup: Adding tests, refactoring, fixing old bugs.

Additional instructions available here The commands available for adding these labels are documented here

[jdumars]

@nikhiljindal @piosz this is a serious impact on the 1.8 release. We need to understand exactly what the risk is of shipping without this test in place, as well as some evidence that the test is stable.

[nikhiljindal]

Sorry am not as familiar with this feature. I will let @piosz comment on that.

[piosz]

@jdumars apologies for the delay but I'm fighting with a cold. I'm working on them, hopefully will finish it today. The original plan was to test it through HPA e2e tests, but we decided to not use metrics-server by default in HPA (see #52553).

[liggitt]

if it is not enabled by default for HPA in 1.8, then I don't think the e2e tests are release-blocking

[ericchiang]

Commented on the feature repo but the release team has concerns about adding e2e tests this close to the release kubernetes/enhancements#271 (comment)

I agree with @liggitt. If this isn't on by default we should remove this from the milestone.

[piosz]

The feature is enabled by default but no other component uses it. The rollout plan is as follow:

• in 1.8 metrics-server was introduced
• in 1.9 we switch HPA to use it by default

I agree that having e2e tests is not a release blocker as HPA doesn't use it.

[luxas]

The feature is enabled by default

To clarify, in GKE only AFAIK, right?

[piosz]

GCE/GKE

[piosz]

fixed in #53205
(we test metrics-server in HPA e2e tests which uses metrics from metrics-server)