Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure test results are archived when test suites fail #7647

Merged
merged 4 commits into from
Sep 5, 2022

Conversation

crazytonyli
Copy link
Contributor

@crazytonyli crazytonyli commented Sep 5, 2022

Description

The buildkite script stops running after test suite fails, but we still need the test result, well especially when any tests failed. This PR took the approach used in WordPress-iOS repo

Testing instructions

Make sure test results are archived when test failed. I'll update this description with screenshots of buildkite steps later.

Screen Shot 2022-09-05 at 1 07 39 PM

Screenshots

N/A


  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

@crazytonyli crazytonyli added the category: tooling Anything that involves building & maintaining the project, including scripts, `Fastfile`, etc. label Sep 5, 2022
@crazytonyli crazytonyli added this to the 10.3 milestone Sep 5, 2022
@crazytonyli crazytonyli requested a review from a team September 5, 2022 00:09
@crazytonyli crazytonyli self-assigned this Sep 5, 2022
@crazytonyli crazytonyli requested a review from a team as a code owner September 5, 2022 00:09
@wpmobilebot
Copy link
Collaborator

wpmobilebot commented Sep 5, 2022

You can test the changes from this Pull Request by:
  • Clicking here or scanning the QR code below to access App Center
  • Then installing the build number pr7647-0dcfec0 on your iPhone

If you need access to App Center, please ask a maintainer to add you.


echo "--- 📦 Zipping test results"
cd fastlane/test_output/ && zip -rq WooCommerce.xcresult.zip WooCommerce.xcresult && cd -

exit $TESTS_EXIT_STATUS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of also adding the echo "^^^ +++" on non-zero exit code (like is done here) so that Buildkite will have the "Testing" section expanded and open by default — instead of the "Zipping test results" one — when the test failed, to avoid confusion for anyone checking the logs and seeing the failure looking like it was on the zipping instead of the testing step?

See Buildkite docs here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea. I've copied this feature over in 0dcfec0

@peril-woocommerce
Copy link

Warnings
⚠️ This PR is assigned to a milestone which is closing in less than 4 days Please, make sure to get it merged by then or assign it to a later expiring milestone

Generated by 🚫 dangerJS

@crazytonyli crazytonyli merged commit a016e82 into trunk Sep 5, 2022
@crazytonyli crazytonyli deleted the save-test-result-as-artifacts branch September 5, 2022 23:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: tooling Anything that involves building & maintaining the project, including scripts, `Fastfile`, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants