Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

suite: fix test failures #1421

Merged
merged 2 commits into from
Oct 6, 2024

Conversation

stevenh
Copy link
Contributor

@stevenh stevenh commented Jul 12, 2023

Summary

Fix TestFailfastSuite when run with go test flag -count=X where X greater than 1.

Motivation

So we don't get false test failures causing confusion.

@dolmen dolmen added test Changes Requested pkg-suite Change related to package testify/suite labels Jul 25, 2023
@dolmen dolmen changed the title fix(suite): test failures suite: fix test failures Oct 16, 2023
@dolmen
Copy link
Collaborator

dolmen commented Oct 16, 2023

This change is conflicting with #1471.

@dolmen dolmen added internal/refactor Refactor internals with no external visible changes internal/testing Changes purely related to the testify testsuites labels Oct 16, 2023
Fix TestFailfastSuite when run with go test flag -count=X where X
greater than 1.
@stevenh
Copy link
Contributor Author

stevenh commented Sep 9, 2024

@dolmen sorry missed the notification for this, conflict fixed.

Copy link
Collaborator

@brackendawson brackendawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super happy with if failFast { having an else but it was already there.

This looks to me like the most appropriate fix for the issue. Down to us running testing.RunTests.

We could avoid a lot of the acrobatics by replacing callOrder with a map or set of booleans to indicate whether test B was called, but then we lose the testing that setup and teardown weren't called for it.

@brackendawson brackendawson merged commit 2fc4e39 into stretchr:master Oct 6, 2024
8 checks passed
@stevenh stevenh deleted the fix/suite-test-failures branch October 6, 2024 21:31
apricote pushed a commit to hetznercloud/fleeting-plugin-hetzner that referenced this pull request Nov 25, 2024
…nercloud/fleeting-plugin-hetzner!175)

This MR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/stretchr/testify](https://github.com/stretchr/testify) | require | minor | `v1.9.0` -> `v1.10.0` |

---

### Release Notes

<details>
<summary>stretchr/testify (github.com/stretchr/testify)</summary>

### [`v1.10.0`](https://github.com/stretchr/testify/releases/tag/v1.10.0)

[Compare Source](stretchr/testify@v1.9.0...v1.10.0)

#### What's Changed

##### Functional Changes

-   Add PanicAssertionFunc by [@&#8203;fahimbagar](https://github.com/fahimbagar) in stretchr/testify#1337
-   assert: deprecate CompareType by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1566
-   assert: make YAML dependency pluggable via build tags by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1579
-   assert: new assertion NotElementsMatch by [@&#8203;hendrywiranto](https://github.com/hendrywiranto) in stretchr/testify#1600
-   mock: in order mock calls by [@&#8203;ReyOrtiz](https://github.com/ReyOrtiz) in stretchr/testify#1637
-   Add assertion for NotErrorAs by [@&#8203;palsivertsen](https://github.com/palsivertsen) in stretchr/testify#1129
-   Record Return Arguments of a Call by [@&#8203;jayd3e](https://github.com/jayd3e) in stretchr/testify#1636
-   assert.EqualExportedValues: accepts everything by [@&#8203;redachl](https://github.com/redachl) in stretchr/testify#1586

##### Fixes

-   assert: make tHelper a type alias by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1562
-   Do not get argument again unnecessarily in Arguments.Error() by [@&#8203;TomWright](https://github.com/TomWright) in stretchr/testify#820
-   Fix time.Time compare by [@&#8203;myxo](https://github.com/myxo) in stretchr/testify#1582
-   assert.Regexp: handle \[]byte array properly by [@&#8203;kevinburkesegment](https://github.com/kevinburkesegment) in stretchr/testify#1587
-   assert: collect.FailNow() should not panic by [@&#8203;marshall-lee](https://github.com/marshall-lee) in stretchr/testify#1481
-   mock: simplify implementation of FunctionalOptions by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1571
-   mock: caller information for unexpected method call by [@&#8203;spirin](https://github.com/spirin) in stretchr/testify#1644
-   suite: fix test failures by [@&#8203;stevenh](https://github.com/stevenh) in stretchr/testify#1421
-   Fix issue [#&#8203;1662](stretchr/testify#1662) (comparing infs should fail) by [@&#8203;ybrustin](https://github.com/ybrustin) in stretchr/testify#1663
-   NotSame should fail if args are not pointers [#&#8203;1661](stretchr/testify#1661) by [@&#8203;sikehish](https://github.com/sikehish) in stretchr/testify#1664
-   Increase timeouts in Test_Mock_Called_blocks to reduce flakiness in CI by [@&#8203;sikehish](https://github.com/sikehish) in stretchr/testify#1667
-   fix: compare functional option names for indirect calls by [@&#8203;arjun-1](https://github.com/arjun-1) in stretchr/testify#1626

##### Documantation, Build & CI

-   .gitignore: ignore "go test -c" binaries by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1565
-   mock: improve doc by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1570
-   mock: fix FunctionalOptions docs by [@&#8203;snirye](https://github.com/snirye) in stretchr/testify#1433
-   README: link out to the excellent testifylint by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1568
-   assert: fix typo in comment by [@&#8203;JohnEndson](https://github.com/JohnEndson) in stretchr/testify#1580
-   Correct the EventuallyWithT and EventuallyWithTf example by [@&#8203;JonCrowther](https://github.com/JonCrowther) in stretchr/testify#1588
-   CI: bump softprops/action-gh-release from 1 to 2 by [@&#8203;dependabot](https://github.com/dependabot) in stretchr/testify#1575
-   mock: document more alternatives to deprecated AnythingOfTypeArgument by [@&#8203;dolmen](https://github.com/dolmen) in stretchr/testify#1569
-   assert: Correctly document EqualValues behavior by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1593
-   fix: grammar in godoc by [@&#8203;miparnisari](https://github.com/miparnisari) in stretchr/testify#1607
-   .github/workflows: Run tests for Go 1.22 by [@&#8203;HaraldNordgren](https://github.com/HaraldNordgren) in stretchr/testify#1629
-   Document suite's lack of support for t.Parallel by [@&#8203;brackendawson](https://github.com/brackendawson) in stretchr/testify#1645
-   assert: fix typos in comments by [@&#8203;alexandear](https://github.com/alexandear) in stretchr/testify#1650
-   mock: fix doc comment for NotBefore by [@&#8203;alexandear](https://github.com/alexandear) in stretchr/testify#1651
-   Generate better comments for require package by [@&#8203;Neokil](https://github.com/Neokil) in stretchr/testify#1610
-   README: replace Testify V2 notice with [@&#8203;dolmen](https://github.com/dolmen)'s V2 manifesto by [@&#8203;hendrywiranto](https://github.com/hendrywiranto) in stretchr/testify#1518

#### New Contributors

-   [@&#8203;fahimbagar](https://github.com/fahimbagar) made their first contribution in stretchr/testify#1337
-   [@&#8203;TomWright](https://github.com/TomWright) made their first contribution in stretchr/testify#820
-   [@&#8203;snirye](https://github.com/snirye) made their first contribution in stretchr/testify#1433
-   [@&#8203;myxo](https://github.com/myxo) made their first contribution in stretchr/testify#1582
-   [@&#8203;JohnEndson](https://github.com/JohnEndson) made their first contribution in stretchr/testify#1580
-   [@&#8203;JonCrowther](https://github.com/JonCrowther) made their first contribution in stretchr/testify#1588
-   [@&#8203;miparnisari](https://github.com/miparnisari) made their first contribution in stretchr/testify#1607
-   [@&#8203;marshall-lee](https://github.com/marshall-lee) made their first contribution in stretchr/testify#1481
-   [@&#8203;spirin](https://github.com/spirin) made their first contribution in stretchr/testify#1644
-   [@&#8203;ReyOrtiz](https://github.com/ReyOrtiz) made their first contribution in stretchr/testify#1637
-   [@&#8203;stevenh](https://github.com/stevenh) made their first contribution in stretchr/testify#1421
-   [@&#8203;jayd3e](https://github.com/jayd3e) made their first contribution in stretchr/testify#1636
-   [@&#8203;Neokil](https://github.com/Neokil) made their first contribution in stretchr/testify#1610
-   [@&#8203;redachl](https://github.com/redachl) made their first contribution in stretchr/testify#1586
-   [@&#8203;ybrustin](https://github.com/ybrustin) made their first contribution in stretchr/testify#1663
-   [@&#8203;sikehish](https://github.com/sikehish) made their first contribution in stretchr/testify#1664
-   [@&#8203;arjun-1](https://github.com/arjun-1) made their first contribution in stretchr/testify#1626

**Full Changelog**: stretchr/testify@v1.9.0...v1.10.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yOC4wIiwidXBkYXRlZEluVmVyIjoiMzkuMjguMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal/refactor Refactor internals with no external visible changes internal/testing Changes purely related to the testify testsuites pkg-suite Change related to package testify/suite test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants