Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix tests in bastion_hosts_test.go file #1754

Merged
merged 1 commit into from
Oct 19, 2021
Merged

Fix tests in bastion_hosts_test.go file #1754

merged 1 commit into from
Oct 19, 2021

Conversation

karuppiah7890
Copy link
Contributor

/kind bug
/kind cleanup
/kind failing-test
/kind flake

What this PR does / why we need it:

Fixes #1753

This PR fixes some of the tests in bastion_hosts_test.go file

f737685 commit made some changes to how bastionhosts service Reconcile method works and removed the creation of public IP from the bastionhosts service's Reconcile logic and moved it into publicips service Reconcile logic by just defining a public ip spec for the bastion host in the cluster scope. The tests were not changed to as part of this commit though, to reflect the changed code. And for some reason the tests passed, though the tests should have ideally failed due to the code change but no test change. This shows there's some shady behavior that we are yet to uncover, leading to false results, in this case false passing tests when the tests should have failed

This PR does the following

  • Remove create publicip fails test in bastion host Reconcile test as we don't create public IP as part of bastion host Reconcile
  • Remove unnecessary / wrong method call expectations using mocks
    • mPublicIP.CreateOrUpdate method call expectations
    • m.Delete(gomockinternal.AContext(), "my-rg", "my-bastionhost1") expectation
  • Fix the order of method call expectations using mocks
    • mPublicIP.Get gets called before mSubnet.Get
  • Comment out t.Parallel temporarily with details about the removal of parallelization - tldr; enabling parallel tests seems to give wrong test results
  • Fix the expected error messages

Which issue(s) this PR fixes:
Fixes #1753

Special notes for your reviewer:

TODOs:

  • squashed commits
  • includes documentation
  • adds modifies unit tests

Release note:

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. kind/flake Categorizes issue or PR as related to a flaky test. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 4, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @karuppiah7890. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added area/provider/azure Issues or PRs related to azure provider sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 4, 2021
@karuppiah7890 karuppiah7890 mentioned this pull request Oct 5, 2021
Closed
3 tasks
@CecileRobertMichon
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 5, 2021
@karuppiah7890
Copy link
Contributor Author

/test pull-cluster-api-provider-azure-e2e

@CecileRobertMichon CecileRobertMichon mentioned this pull request Oct 7, 2021
Closed
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 8, 2021
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 15, 2021
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 15, 2021
@karuppiah7890
Copy link
Contributor Author

Sorry about the messy diff. I reordered the tests to put failure scenarios first and then finally the success scenario, for both Reconcile and Delete test

@karuppiah7890
fix tests in bastion_hosts_test.go file
bcfe2cd 
- Remove `create publicip fails` test in bastion host `Reconcile` test as we don't create public IP as part of bastion host `Reconcile`
- Remove `bastion successfully created with created public ip` test in bastion host `Reconcile` test as we don't create public IP as part of bastion host `Reconcile`
- Remove unnecessary / wrong method call expectations using mocks
  - `mPublicIP.CreateOrUpdate` method call expectations
  - `m.Delete(gomockinternal.AContext(), "my-rg", "my-bastionhost1")` expectation
- Fix the order of method call expectations using mocks
  - `mPublicIP.Get` gets called before `mSubnet.Get`
- Comment out `t.Parallel` temporarily with details about the removal of parallelization - tldr; enabling parallel tests seems to give wrong test results
- Fix the expected error messages

Signed-off-by: Karuppiah Natarajan <[email protected]>
devigned
devigned approved these changes Oct 15, 2021
View reviewed changes
Copy link
Contributor

@devigned devigned left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/assign @CecileRobertMichon

Much better!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 15, 2021
mboersma
mboersma approved these changes Oct 15, 2021
View reviewed changes
Copy link
Contributor

@mboersma mboersma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

azure/services/bastionhosts/bastionhosts_test.go
)
},
},
}

for _, tc := range testcases {
tc := tc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find!

@karuppiah7890 karuppiah7890 mentioned this pull request Oct 17, 2021
Merged
7 tasks
CecileRobertMichon
CecileRobertMichon reviewed Oct 18, 2021
View reviewed changes
Copy link
Contributor

@CecileRobertMichon CecileRobertMichon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 18, 2021
@k8s-ci-robot k8s-ci-robot merged commit 3ef19e7 into kubernetes-sigs:main Oct 19, 2021
@k8s-ci-robot k8s-ci-robot added this to the v0.5 milestone Oct 19, 2021
@karuppiah7890 karuppiah7890 deleted the fix-1753 branch October 19, 2021 03:16
@CecileRobertMichon CecileRobertMichon modified the milestones: v0.5, v1.0 Oct 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CecileRobertMichon CecileRobertMichon CecileRobertMichon left review comments

@devigned devigned devigned approved these changes

@mboersma mboersma mboersma approved these changes

@alexeldeib alexeldeib Awaiting requested review from alexeldeib

Assignees

@mboersma mboersma

@devigned devigned

@CecileRobertMichon CecileRobertMichon

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/provider/azure Issues or PRs related to azure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. kind/flake Categorizes issue or PR as related to a flaky test. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.0
Development

Successfully merging this pull request may close these issues.

bastionhosts_test.go tests give false results when running tests in parallel
5 participants
@karuppiah7890 @k8s-ci-robot @CecileRobertMichon @mboersma @devigned