Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swift test output in xunit format reports skipped tests as successful #8066

Open
1 task done
jozefizso opened this issue Oct 22, 2024 · 0 comments
Open
1 task done
Labels

Comments

@jozefizso
Copy link

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

  • Confirmed reproduction steps with SwiftPM CLI. The description text must include reproduction steps with either of command-line SwiftPM commands, swift build, swift test, swift package etc.

Description

The swift test has option to generate xunit report from the test run. The resulting XML does not include skipped tests and they are attributed as successful instead.

This is misleading because those tests never run and cannot be accounted for as working.

Expected behavior

The XCTest which are marked as skipped should be reported in the xunit output as skipped.

Actual behavior

The XCTests which are marked as skipped are reported in the xunit format as successful.

Steps to reproduce

  1. Create Swift package
  2. Add XCTest based tests
  3. Write a test which is skipped (it throws the XCSkip error)
  4. Run the tests and get xunit output using command:
swift test --parallel --xunit-output 'TestReport.xml'

Observe the XML output. The skipped test is marked as successful.

Swift Package Manager version/commit hash

No response

Swift & OS version (output of swift --version ; uname -a)

swift-driver version: 1.90.11.1 Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4)
Target: arm64-apple-macosx14.0

@jozefizso jozefizso added the bug label Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant