Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): use nextests groups to isolate some tests #2617

Conversation

divagant-martian
Copy link
Contributor

Description

After a lot of debugging we realized the integrations tests (gc tests, for
example) run with swarm discovery on since they are not technically using test
iroh, so they bypass the cfg that prevents it from being turned on. This causes
the test_local_swarm_discovery test to be flooded with nodes from other
tests, making it flaky. Isolating the test should make it no longer flaky
anymore

Breaking Changes

n/a

Notes & open questions

We should still create a test that creates a bunch of nodes and makes sure they
all can find each other within reasonable time. There is still reason to
believe swarm discovery is slow, particularly in linux, but we should address
it in a test under our countrol, not via the non-deterministic nature of
concurrent tests in ci.

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.

Copy link
Collaborator

@Arqu Arqu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree on the tests to be added.

This looks to work as intended. Ran cargo nextest show-config test-groups to confirm and got what was expected:

 Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 35s
group: run-in-isolation (max threads = 1)
  * override for default profile with filter 'test(::run_in_isolation::)':
      iroh-net:
          discovery::local_swarm_discovery::tests::run_in_isolation::test_local_swarm_discovery
          ```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, this cool. maybe there are other things we could use this for. A bit of a shame the only kind of marker you have is the name, but better than nothing.

@divagant-martian divagant-martian added this pull request to the merge queue Aug 13, 2024
Merged via the queue into n0-computer:main with commit a5072c3 Aug 13, 2024
46 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

flaky test: iroh-net discovery::local_swarm_discovery::tests::test_local_swarm_discovery
3 participants