Add GitHub Action to report test failures in the workflow summary

[jamesnetherton]

Adds a custom action to report test failures into the workflow summary page.

I added a commit that will cause some failures so it can be tested. I'll drop it before merging.

[jamesnetherton]

Test failures are listed here:

https://github.com/apache/camel-quarkus/actions/runs/8997456627.

[zhfeng]

@jamesnetherton Great work!

Just wonder if it is possible to add such a comment in PR automatically?

[jamesnetherton]

Just wonder if it is possible to add such a comment in PR automatically?

Yeah, I did think about that. Not sure if we'd maybe run into permissions issues (needs checking)?

I also didn't want to force the PR comment requirement, since on the nightly workflows, there is no PR.

I can try to investigate it.

[jamesnetherton]

Just wonder if it is possible to add such a comment in PR automatically?

Another reason I didn't do it, is that it makes things a lot more complicated.

For it to work reliably and avoid concurrency issues, you need to capture all of the test reports from each build job. Then at the end of the build, recover them and parse out the detail.

Which is why I'm thinking of merging the current implementation and improving it later.

[zhfeng]

Yeah, I understand. What I need currently is just a comment with a link to the summary when there is a test failure. This could be much easier?

[jamesnetherton]

This could be much easier?

Yes & no 🙂

I could try to add something. I have a feeling that it wont work with PRs from forked repositories though. It'd need a bot user token.

[zhfeng]

Hmm, no, that's definely a problem. :(

[jamesnetherton]

How should I proceed?

Merge as-is and try to follow up later with improvements? Or rethink things entirely?

Even something as simple as a comment with a link is non-trivial to manage. They get stale, so need updating on future workflow runs. Or the tests pass and they become irrelevant so need minimising or deleting.

I deliberately kept the initial scope of this to be as simple as possible. It's two clicks from the PR to get to the summary. So not a massive overhead (IMO).

[zhfeng]

I'd say let's merge and keep to improve it. Thanks @jamesnetherton !