Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test #2395

Merged
merged 3 commits into from
Apr 21, 2018
Merged

Fix flaky test #2395

merged 3 commits into from
Apr 21, 2018

Conversation

ElvinEfendi
Copy link
Member

What this PR does / why we need it:
The PR fixes flakiness with dynamic configuration tests and shaves off some seconds from the test run.

Which issue this PR fixes: n/a

Special notes for your reviewer: n/a

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 21, 2018
@@ -52,8 +52,6 @@ var _ = framework.IngressNginxDescribe("Dynamic Configuration", func() {
Expect(err).NotTo(HaveOccurred())
Expect(ing).NotTo(BeNil())

time.Sleep(10 * time.Second)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not necessary because WaitForNginxServer below will make sure we wait enough after ensureIngress for the logs to appear.

@@ -88,6 +86,7 @@ var _ = framework.IngressNginxDescribe("Dynamic Configuration", func() {
replicas := 2
err := framework.UpdateDeployment(f.KubeClientSet, f.IngressController.Namespace, "http-svc", replicas, nil)
Expect(err).NotTo(HaveOccurred())
time.Sleep(5 * time.Second)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to give some time for the logs to appear, 5 seconds might be a bit too conservative but we can revisit this later

log, err := f.NginxLogs()
Expect(err).ToNot(HaveOccurred())
Expect(log).ToNot(BeEmpty())
index := strings.Index(log, fmt.Sprintf("reason: 'UPDATE' Ingress %s/foo.com", f.IngressController.Namespace))
index := strings.Index(log, "id=should_handle_annotation_changes")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the actual fix. I noticed that marking the start of logs using reason: 'UPDATE' Ingress %s/foo.com is not enough because some logs appear before that even though those logs are result of that update.

So now we mark the start of the logs we are interested in using a request with unique query string to this test.

@@ -153,7 +159,14 @@ var _ = framework.IngressNginxDescribe("Dynamic Configuration", func() {
_, err = f.KubeClientSet.ExtensionsV1beta1().Ingresses(f.IngressController.Namespace).Update(ingress)
Expect(err).ToNot(HaveOccurred())

time.Sleep(5 * time.Second)
By("generating the respective ssl listen directive")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by moving WaitForNginxServer we can get rid of sleep and let WaitForNginxServer to take care of it

@@ -193,12 +198,10 @@ var _ = framework.IngressNginxDescribe("Dynamic Configuration", func() {
}
_, err = f.KubeClientSet.ExtensionsV1beta1().Ingresses(f.IngressController.Namespace).Update(ingress)
Expect(err).ToNot(HaveOccurred())
time.Sleep(5 * time.Second)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already sleep below, this is not necessary

@@ -290,8 +293,6 @@ func enableDynamicConfiguration(namespace string, kubeClientSet kubernetes.Inter
return err
}

time.Sleep(5 * time.Second)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was probably some leftover from previous implementation - I don't see any reason for it to stay now.

@ElvinEfendi ElvinEfendi changed the title Fix flaky test [WIP] Fix flaky test Apr 21, 2018
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 21, 2018
@codecov-io
Copy link

codecov-io commented Apr 21, 2018

Codecov Report

Merging #2395 into master will increase coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2395      +/-   ##
==========================================
+ Coverage    39.3%   39.32%   +0.01%     
==========================================
  Files          73       73              
  Lines        5216     5216              
==========================================
+ Hits         2050     2051       +1     
+ Misses       2878     2876       -2     
- Partials      288      289       +1
Impacted Files Coverage Δ
internal/watch/file_watcher.go 84.61% <0%> (+3.84%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7625c05...0837e81. Read the comment docs.

@ElvinEfendi ElvinEfendi changed the title [WIP] Fix flaky test Fix flaky test Apr 21, 2018
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 21, 2018
@ElvinEfendi
Copy link
Member Author

/assign aledbf

@aledbf
Copy link
Member

aledbf commented Apr 21, 2018

/approve

@aledbf
Copy link
Member

aledbf commented Apr 21, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 21, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aledbf, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit f1be6d0 into kubernetes:master Apr 21, 2018
@ElvinEfendi ElvinEfendi deleted the fix-flaky-test branch April 21, 2018 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants