Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Omit block for test failure details in annotation when there's none #63

Merged
merged 4 commits into from
Jun 8, 2023

Conversation

mokagio
Copy link
Contributor

@mokagio mokagio commented Jun 5, 2023

Not all test nodes failures in the JUnit report have a value.

E.g. from the failures in this build:

<testcase classname="StatsTests" name="testInsightsStatsLoadProperly()" time="131.52018702030182">
  <failure message="failed: caught error: "timedOut" (#CharacterRangeLen=0)"> </failure>
</testase>

In such cases, the failure annotation looks a bit odd:

image

The changes in this PR make the annotation look like this:

image


  • I have considered if this change warrants release notes and have added them to the appropriate section in the CHANGELOG.md if necessary.

@mokagio mokagio force-pushed the mokagio/omit-details-annotation-when-empty branch from 55ad92f to 7216c19 Compare June 5, 2023 05:57
@mokagio mokagio requested a review from a team June 5, 2023 10:07
Comment on lines +86 to +87
<pre>#{@message}</pre>
#{formatted_details}
Copy link
Contributor Author

@mokagio mokagio Jun 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could be convinced that the <pre> makes sense only when there are no details.

In case both failure and details have a value, we'd get something like:


failure name

details details
details

That might not be the best to look at 🤔

Then again, we don't have to look at that info for long 😅


Annotates the Buildkite build with a summary of failed and flaky tests based on a JUnit report file (defaults to using `build/results/report.junit`).
Optionally also posts the same info to a Slack channel.
HELP
opts.on('--slack CHANNEL_NAME', 'The name of the Slack channel to post the failure summary to') { |v| slack_channel = '#' + v.delete_prefix('#') }
opts.on('--slack CHANNEL_NAME', 'The name of the Slack channel to post the failure summary to') { |v| slack_channel = "##{v.delete_prefix('#')}" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: I find three usages of # here a bit confusing. Not sure if this suggestion would make it better or worse 😄

Suggested change
opts.on('--slack CHANNEL_NAME', 'The name of the Slack channel to post the failure summary to') { |v| slack_channel = "##{v.delete_prefix('#')}" }
opts.on('--slack CHANNEL_NAME', 'The name of the Slack channel to post the failure summary to') { |v| slack_channel = v.delete_prefix('#').prepend('#') }

@mokagio
Copy link
Contributor Author

mokagio commented Jun 8, 2023

We've been wondering with how to best test these changes and other changes. See internal ref p1685981991334859-slack-C02KLTL3MKM

It's crucial for the long term health of this project that we find an answer. However, I think the scope is broader than this PR and therefore I'm going to merge in the meantime to roll out the improvement.

On that note, I had a play around and have a proof of concept for using RSpec to test part of the code edited here: #64

@mokagio mokagio merged commit c252bcf into trunk Jun 8, 2023
@mokagio mokagio deleted the mokagio/omit-details-annotation-when-empty branch June 8, 2023 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants