Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨introduce test E2E #2884

Merged
merged 1 commit into from
Apr 17, 2020

Conversation

fabriziopandini
Copy link
Member

@fabriziopandini fabriziopandini commented Apr 8, 2020

What this PR does / why we need it:
This PR introduces an E2e folder that introduces clusterctl based E2E test for quick-start. self-hosted, KCP upgrade.

Most notably:

  • Such test can be executed with any infrastructure provider (not only CAPD)
  • Test spec are exposed as a library so they can be reused by other projects

Which issue(s) this PR fixes:
xref #2753
xref #2637
xref #2636

/assign @vincepri
/assign @sedefsavas

@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 8, 2020
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 8, 2020
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 8, 2020
test/e2e/e2e_suite_test.go Outdated Show resolved Hide resolved
Namespace: namespaceName,
})
Expect(controlPlane).ToNot(BeNil())
framework.WaitForOneKubeadmControlPlaneMachineToExist(context.TODO(), framework.WaitForOneKubeadmControlPlaneMachineToExistInput{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to wait for all machines to be created before the CNI

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a particular reason to do so?
AFAIK installing as soon as the control plane is up is legit; it is also a precondition to get the control plane ready...

test/e2e/quick_start_test.go Outdated Show resolved Hide resolved
@wfernandes
Copy link
Contributor

/assign
Assigning myself to this because I'd like to get context on this PR and help write other tests.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 10, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 13, 2020
test/e2e/Makefile Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 15, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 15, 2020
@fabriziopandini fabriziopandini force-pushed the e2e-quickstart branch 2 times, most recently from 2d905fd to a353b37 Compare April 16, 2020 16:20
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 16, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 16, 2020
@fabriziopandini fabriziopandini changed the title [WIP] ✨quick start E2E ✨quick start E2E Apr 16, 2020
@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

@wfernandes
Copy link
Contributor

Reviewing

@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

@@ -71,6 +69,8 @@ func KCPUpgradeSpec(ctx context.Context, inputGetter func() KCPUpgradeSpecInput)
It("Should successfully upgrade Kubernetes, DNS, kube-proxy, and etcd", func() {

By("Creating a workload cluster")
Expect(input.E2EConfig.Variables).To(HaveKey(clusterctl.KubernetesVersion))
Expect(input.E2EConfig.Variables).To(HaveKey(clusterctl.CNIPath))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need these checks for every test? We already throw error if they are empty during validation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should enforce variables to exist during validation... I missed this.
Each spec might require different variables

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KubernetesVersion and CNI should exist for all tests? If not we should remove those checks from validation.

@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

@sedefsavas
Copy link

Changes look good to me. Great work @fabriziopandini.
Over to @wfernandes and @vincepri.

@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

@vincepri
Copy link
Member

/milestone v0.3.4

@k8s-ci-robot k8s-ci-robot added this to the v0.3.4 milestone Apr 17, 2020
Copy link
Member

@vincepri vincepri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini, vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 17, 2020
Copy link
Contributor

@wfernandes wfernandes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed a couple of things regarding contexts. But this is great work! Seems like we even have three e2e tests done!

test/e2e/kcp_upgrade.go Outdated Show resolved Hide resolved
test/e2e/kcp_upgrade.go Outdated Show resolved Hide resolved
test/e2e/kcp_upgrade.go Outdated Show resolved Hide resolved
test/e2e/self_hosted.go Show resolved Hide resolved
@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

@fabriziopandini
Copy link
Member Author

@wfernandes thanks for the feedback!
comment addressed

@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

Verified

This commit was signed with the committer’s verified signature.
ncdc Andy Goldstein
@fabriziopandini
Copy link
Member Author

/test pull-cluster-api-e2e

@k8s-ci-robot
Copy link
Contributor

@fabriziopandini: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
pull-cluster-api-apidiff 5596ab2 link /test pull-cluster-api-apidiff

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@fabriziopandini
Copy link
Member Author

@wfernandes @sedefsavas for final lgtm
the job now runs in parallel and we are down to 20m for 3 test, which is not bad vs 40m that capd-e2e takes for 2 only

@wfernandes
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 17, 2020
@k8s-ci-robot k8s-ci-robot merged commit 2481422 into kubernetes-sigs:master Apr 17, 2020
@fabriziopandini fabriziopandini deleted the e2e-quickstart branch April 18, 2020 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants