refactor: go test -short does not use the host network #1352
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This diff refactors the codebase so that we avoid using the host network when running `go test -short`.
bassosimone
commented
Oct 9, 2023
bassosimone
commented
Oct 9, 2023
26 tasks
Murphy-OrangeMud
pushed a commit
to Murphy-OrangeMud/probe-cli
that referenced
this pull request
Feb 13, 2024
This diff refactors the codebase so that we avoid using the host network when running `go test -short`. This change is good in general, because now coverage tells us the amount of code we're covering without depending on interactions with an existing network, which means these tests behave in the same way in ~any place. I expect a coverage drop from this PR, because there's some coverage made with integration testing (if we consider integration tests the tests that require the host network interface with uncensored internet access). To make this happen, I needed to modify the `quictesting` package (now moved to toplevel and renamed `testingquic`) such that it attempts to get a known-to-work-well endpoint for QUIC _only_ when the developer using the package really needs it, rather than on import. Before doing this, there were several tests that panicked because `quictesting` could not figure out which IP address to use when you disable the WiFi or run inside another netns. Now we only figure this IP address out the first time a test requires us to give it either the domain or the endpoint that we should use. To be sure we continue to honour the promise that `go test -short` does not use the host network, I needed to refactor the CI such that we measure coverage inside a new network namespace with only localhost support. I think this compromise is acceptable, since the original ask was to avoid flaky network tests (see ooni/probe#2426). Because of this change in how we run the coverage checks, I am tentatively enabling also running all tests for pull requests, otherwise we don't know if a contribution breaks tests using the network. Hopefully, we should be fine because we are caching previous runs, so a bunch of tests should already be cached. Closes ooni/probe#2426. While there, enable again some backend integration tests, and close ooni/probe#2539.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This diff refactors the codebase so that we avoid using the host network when running
go test -short. This change is good in general, because now coverage tells us the amount of code we're covering without depending on interactions with an existing network, which means these tests behave in the same way in ~any place.I expect a coverage drop from this PR, because there's some coverage made with integration testing (if we consider integration tests the tests that require the host network interface with uncensored internet access).
To make this happen, I needed to modify the
quictestingpackage (now moved to toplevel and renamedtestingquic) such that it attempts to get a known-to-work-well endpoint for QUIC only when the developer using the package really needs it, rather than on import. Before doing this, there were several tests that panicked becausequictestingcould not figure out which IP address to use when you disable the WiFi or run inside another netns. Now we only figure this IP address out the first time a test requires us to give it either the domain or the endpoint that we should use.To be sure we continue to honour the promise that
go test -shortdoes not use the host network, I needed to refactor the CI such that we measure coverage inside a new network namespace with only localhost support. I think this compromise is acceptable, since the original ask was to avoid flaky network tests (see ooni/probe#2426).Because of this change in how we run the coverage checks, I am tentatively enabling also running all tests for pull requests, otherwise we don't know if a contribution breaks tests using the network. Hopefully, we should be fine because we are caching previous runs, so a bunch of tests should already be cached.
Closes ooni/probe#2426.
While there, enable again some backend integration tests, and close ooni/probe#2539.