Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Integration Test Suite Resilience #273

Merged
merged 2 commits into from
Mar 2, 2024

Conversation

zombieobject
Copy link
Contributor

Introduction 📟

This pull request improves the resilience and readability of the Integration Test Suite

Purpose ℹ️

  • Resiliency Improvements:
    • Given test runner resource limitations and external dependencies for the test suite, a retry loop is needed.
  • Readability improvements:
    • decreased test log verbosity
    • grep statements added for final test results

Discussion 🎙

  • Once "warp" runners are made available again and tested, the retry count will likely be reduced.
  • Further integrations of interactive swift-lint and swift-format during SDK local builds to follow.

@zombieobject zombieobject self-assigned this Mar 2, 2024
@zombieobject zombieobject requested a review from a team as a code owner March 2, 2024 06:43
@zombieobject zombieobject enabled auto-merge (squash) March 2, 2024 06:59
@zombieobject zombieobject force-pushed the em/integration-resilience branch from d64353b to 4167f98 Compare March 2, 2024 07:21
Copy link
Contributor

@neekolas neekolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to dig in to why these tests are so flaky. Other SDKs use the same backend and don't have the same flakiness issues.

@zombieobject zombieobject merged commit 0e4e79c into main Mar 2, 2024
2 checks passed
@zombieobject zombieobject deleted the em/integration-resilience branch March 2, 2024 08:27
@zombieobject
Copy link
Contributor Author

zombieobject commented Mar 2, 2024

I'd love to dig in to why these tests are so flaky. Other SDKs use the same backend and don't have the same flakiness issues.

Same here! While depending on a back end is counter to most testing architectures on mobile, aside from E2E tests, the flakiness is an indicator of some larger problem. I suspect threading issues. Either way, resolving this will be one of my many top priorities but will take some time to understand the how and why of the current state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants