Skip to content

Commit

Permalink
Use XML nextest output instead of ad-hoc parsing
Browse files Browse the repository at this point in the history
This eliminates the piping, tricky parsing, and special-casing to
preserve colorization, and runs the tests and the XML parsing in
the default `pwsh` shell (since this is a Windows job), using
PowerShell facilities to parse the XML. This also checks that there
are no *errors*, in addition to (still) checking that there are no
more *failures* than expected.

In the preceding commit, five additional tests, not currently noted
in #1358, failed:

    FAIL [   0.010s] gix-credentials::credentials program::from_custom_definition::empty
    FAIL [   0.008s] gix-credentials::credentials program::from_custom_definition::name
    FAIL [   0.010s] gix-credentials::credentials program::from_custom_definition::name_with_args
    FAIL [   0.009s] gix-credentials::credentials program::from_custom_definition::name_with_special_args
    FAIL [   0.014s] gix-discover::discover upwards::from_dir_with_dot_dot

In addition, one test noted in #1358 does not always fail on CI,
because it is a performance test and the CI runner is fast enough
so that it usually passes:

    FAIL [ 181.270s] gix-ref-tests::refs packed::iter::performance

Let's see if running the tests more similarly to the way they are
run on Windows without `GIX_TEST_IGNORE_ARCHIVES`, i.e. without
piping and with the Windows default of `pwsh` as the shell, affects
any of those new/CI-specific failures.
  • Loading branch information
EliahKagan committed Nov 7, 2024
1 parent 6b6436b commit 145ae49
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 11 deletions.
2 changes: 2 additions & 0 deletions .config/nextest.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[profile.with-xml.junit]
path = "junit.xml"
15 changes: 4 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,6 @@ jobs:

test-fixtures-windows:
runs-on: windows-latest
defaults:
run:
shell: bash # Includes `-o pipefail`.
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
Expand All @@ -96,22 +93,18 @@ jobs:
id: nextest
env:
GIX_TEST_IGNORE_ARCHIVES: 1
run: |
cargo nextest --color=always run --workspace --no-fail-fast |& tee nextest.log
run: cargo nextest --profile=with-xml run --workspace --no-fail-fast
continue-on-error: true
- name: Check how many tests failed
if: steps.nextest.outcome == 'failure'
env:
LC_ALL: C.utf8
# FIXME: Change to 15 after verifying that the last step can fail.
# See https://github.com/GitoxideLabs/gitoxide/issues/1358.
EXPECTED_FAILURE_COUNT: 13
run: |
set -x
sed -Ei 's/\x1B\[[[:digit:];]*m//g' nextest.log # Remove ANSI color codes.
pattern='\n-{10,}\r?\n[ \t]+Summary\b[^\n]+[ \t]\K\d+(?= failed\b)'
count="$(grep -zoP "$pattern" nextest.log)"
((count <= EXPECTED_FAILURE_COUNT))
[xml]$junit = Get-Content -Path 'target/nextest/with-xml/junit.xml'
if ($junit.testsuites.errors -ne 0) { exit 1 }
if ($junit.testsuites.failures -gt $env:EXPECTED_FAILURE_COUNT) { exit 1 }
test-32bit:
runs-on: ubuntu-latest
Expand Down

0 comments on commit 145ae49

Please sign in to comment.