Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update integration tests to use a locally built version of kic #7857

Closed
wants to merge 8 commits into from

Conversation

priyawadhwa
Copy link

@priyawadhwa priyawadhwa commented Apr 22, 2020

Depends on #7856 and #7858 to merge first

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 22, 2020
@priyawadhwa priyawadhwa changed the title wip: Update integration tests to use a locall built version of kic wip: Update integration tests to use a locally built version of kic Apr 22, 2020
@priyawadhwa
Copy link
Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 22, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: priyawadhwa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 22, 2020
@codecov-io
Copy link

codecov-io commented Apr 22, 2020

Codecov Report

Merging #7857 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #7857   +/-   ##
=======================================
  Coverage   35.61%   35.61%           
=======================================
  Files         148      148           
  Lines        9303     9303           
=======================================
  Hits         3313     3313           
  Misses       5593     5593           
  Partials      397      397           

@minikube-pr-bot
Copy link

kvm2 Driver
docker Driver

cmd/minikube/cmd/start_flags.go Outdated Show resolved Hide resolved
touch "${TEST_OUT}"
${SUDO_PREFIX}${E2E_BIN} \
-minikube-start-args="--driver=${VM_DRIVER} ${EXTRA_START_ARGS}" \
-minikube-start-args="--driver=${VM_DRIVER} --docker-base-image=kic ${EXTRA_START_ARGS}" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be added to our github actions too
I recommend adding a new job in the 3 jobs groups that are running before integraiton tests
currently they are build_minikube, unit_test, lint
we should add another job called -build build-base image-
and make the follwing integraiton jobs adds
needs: [build-minikube , build-base-image-

@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: [62.053907941 64.194861292 68.058690058]
Average time for minikube: 64.769153097

Times for Minikube (PR 7857): [65.008399247 64.968777433 63.591683065]
Average time for Minikube (PR 7857): 64.52295324833334

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7857) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.0 on   |  0.065486 |           0.063108 |
| Debian 9.11                    |           |                    |
| * Using the kvm2 driver based  |  0.021237 |           0.024173 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003157 |           0.003545 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 41.520677 |          40.946219 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.0 | 20.916798 |          21.096356 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.390215 |           1.449185 |
| components...                  |           |                    |
| * Enabled addons:              |  0.766982 |           0.860829 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.080895 |           0.074248 |
| configured to use "minikube"   |           |                    |
|                                |  0.003706 |           0.005290 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.870350592 27.225572476000007 26.238534075999997]
Average time for minikube: 26.77815238133333

Times for Minikube (PR 7857): [41.36577660999999 26.523642036000002 27.703785408000005]
Average time for Minikube (PR 7857): 31.86440135133333

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7857) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.0 on           |  0.081745 |           0.081326 |
| Debian 9.11                            |           |                    |
| * Using the docker driver              |  0.003240 |           0.002479 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.062579 |           0.060280 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  8.135551 |          11.063658 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.0         |  0.000256 |           0.000185 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.542295 |          19.566038 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  0.881707 |           0.867864 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.002684 |           0.153969 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.063369 |           0.063654 |
| configured to use "minikube"           |           |                    |
|                                        |  0.004725 |           0.004948 |
+----------------------------------------+-----------+--------------------+

Makefile Show resolved Hide resolved
.github/workflows/pr.yml Outdated Show resolved Hide resolved
.github/workflows/pr.yml Outdated Show resolved Hide resolved
@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: [69.49377153399999 65.155327627 65.75245636199999]
Average time for minikube: 66.80051850766665

Times for Minikube (PR 7857): [68.717599767 65.56597774 66.11161530800001]
Average time for Minikube (PR 7857): 66.798397605

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7857) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.1 on   |  0.065437 |                    |
| Debian 9.11                    |           |                    |
| * Using the kvm2 driver based  |  0.020894 |           0.020756 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003456 |           0.003295 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 41.210056 |          40.993319 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.0 | 23.119424 |          23.166319 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.664988 |           1.390250 |
| components...                  |           |                    |
| * Enabled addons:              |  0.639262 |           1.080110 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.073584 |           0.078312 |
| configured to use "minikube"   |           |                    |
|                                |  0.003418 |           0.061702 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.764297668 28.263549069 28.262532844000003]
Average time for minikube: 27.763459860333338

Times for Minikube (PR 7857): [26.10366515 26.208711791 26.890062625]
Average time for Minikube (PR 7857): 26.400813188666664

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7857) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.1 on           |  0.077272 |                    |
| Debian 9.11                            |           |                    |
| * Using the docker driver              |  0.002710 |           0.002626 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.057432 |           0.060435 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  7.881590 |           7.843990 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.0         |  0.000260 |           0.000182 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 18.125299 |          17.007241 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.500727 |           1.180333 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.048199 |           0.144554 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.066489 |           0.077011 |
| configured to use "minikube"           |           |                    |
|                                        |  0.003482 |           0.079109 |
+----------------------------------------+-----------+--------------------+

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 24, 2020
@priyawadhwa priyawadhwa changed the title wip: Update integration tests to use a locally built version of kic Update integration tests to use a locally built version of kic May 1, 2020
@k8s-ci-robot k8s-ci-robot removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels May 1, 2020
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 1, 2020
@minikube-pr-bot
Copy link

kvm2 Driver
Times for minikube: [65.710892213 64.598952185 65.38277810000001]
Average time for minikube: 65.23087416599999

Times for Minikube (PR 7857): [79.271983969 64.97733529700001 66.071959173]
Average time for Minikube (PR 7857): 70.10709281300001

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 7857) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.2 on   |  0.059853 |                    |
| Debian 9.11                    |           |                    |
| * Using the kvm2 driver based  |  0.022288 |           0.023231 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003923 |           0.003282 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 40.052336 |          45.001034 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.1 | 22.562250 |                    |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  1.604052 |           1.644989 |
| components...                  |           |                    |
| * Enabled addons:              |  0.840573 |           0.110407 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.081534 |           0.076158 |
| configured to use "minikube"   |           |                    |
|                                |  0.004065 |           0.003808 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [26.999116901999997 29.029186448 26.943213542999995]
Average time for minikube: 27.657172297666666

Times for Minikube (PR 7857): [27.196796482 27.951698336 28.763457967999997]
Average time for Minikube (PR 7857): 27.970650928666668

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 7857) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.2 on           |  0.083215 |                    |
| Debian 9.11                            |           |                    |
| * Using the docker driver              |  0.002962 |           0.003300 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.066071 |           0.066602 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  8.277296 |           8.137523 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.1         |  0.128810 |                    |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 17.728436 |          18.043588 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.185820 |           1.561131 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.112016 |           0.003415 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.069051 |           0.066290 |
| configured to use "minikube"           |           |                    |
|                                        |  0.003495 |           0.000224 |
+----------------------------------------+-----------+--------------------+

@@ -20,6 +20,8 @@ jobs:
run: |
make minikube-linux-amd64
make e2e-linux-amd64
make kic-base-image
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should only do this only if the Dockerfile been changed. if not it can cause all sort of issues. currently our base ubuntu image is not pinned and each time we get a different installed versions. not good idea for a reliable idea of testing what we ship to the users.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 19, 2020
@k8s-ci-robot
Copy link
Contributor

@priyawadhwa: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@priyawadhwa
this PR still needs to address the comment that it should build the image only if the Dockerfile was changed compared to the master, we dont want to rebuild same Dockerfile on each test (that will result in slightly different packages)

@priyawadhwa
Copy link
Author

closing until i get a chance to look at this again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants