Skip to content

Commit

Permalink
ci: disable fail-fast for Rust version matrix jobs (#640)
Browse files Browse the repository at this point in the history
Tower has test and check jobs on CI that run on a build matrix
including a number of Rust versions. By default, GitHub Actions has
fail-fast semantics for matrix jobs, so if any matrix job fails, the
rest are cancelled and the build is failed. This is intended to help
builds complete faster.

This isn't really the ideal behavior for testing across multiple Rust
versions. When a build fails on a particular toolchain version, we would
ideally like to know whether the failure is localized to that version or
exists on _all_ Rust versions. This is particularly important for builds
on nightly Rust, as the nightly toolchain is more likely to contain
compiler regressions that might not be our fault at all. Similarly, we
might want to know if a change only broke the build on our MSRV, or if
it broke the build everywhere --- such an issue would be fixed
differently.

This also currently means that the nightly test run failing will prevent
PRs from being merged, even if the failure is due to a nightly compiler
regression. We currently only *require* the stable and MSRV test runs
to pass in order to merge a PR, but because the fail-fast behavior
will cancel them if the nightly build fails, this means that nightly failing
will effectively prevent merging PRs...which, given that it's not marked
as required, seems different from what we intended.

Therefore, this PR changes the CI workflow to disable fail-fast behavior
on the cross-version test jobs.

Signed-off-by: Eliza Weisman <[email protected]>
  • Loading branch information
hawkw authored Feb 11, 2022
1 parent 2f50b49 commit 665834c
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ jobs:
check:
runs-on: ubuntu-latest
strategy:
# Disable fail-fast. If the test run for a particular Rust version fails,
# don't cancel the other test runs, so that we can determine whether a
# failure only occurs on a particular version.
fail-fast: false
matrix:
rust: [stable, 1.46.0]
steps:
Expand Down Expand Up @@ -59,6 +63,10 @@ jobs:
needs: check
runs-on: ubuntu-latest
strategy:
# Disable fail-fast. If the test run for a particular Rust version fails,
# don't cancel the other test runs, so that we can determine whether a
# failure only occurs on a particular version.
fail-fast: false
matrix:
rust: [stable, beta, nightly, 1.46.0]
steps:
Expand Down

0 comments on commit 665834c

Please sign in to comment.