When to use Annotations for CI/CD checks

[al-fisher]

Following the continued development and role out of the multiple CI/CD checks - wondering if we revisit where and when we use annotations to provide feedback to developers?

I am thinking there are two conditions:

1. where the submitted code has caused the issue (i.e. issue not on master/main branch and now is on current PR branch!)
2. where the issue relates to a definable line in the code - such that annotation gives good context

Where neither of the two conditions apply, suggest we add all the info to the Checks tab only (and encourage review and viewing there by expert/admin developers) - and do not now add as annotations that appear in the Files changed tab.
This for me will help as the the Checks tab can provide more easily all the necessary context as needed - keeping the Files changed nice and clean for majority of users and focused on suggested and proposed code changes and review comments.

Some of those Annotations are still quite big actually 😄, with different strategy needed to reduce size!
For these legacy and long outstanding issues - that do need visibility and reminders to address - we could log as part of central code quality status dashboard/report instead.

---

To illustrate, I see the documentation-compliance as being a really good example where the annotations, in the summary on the Checks tab, but then hyperlinked to and then repeated in location in the Files changed tab works really well ...

IsInputAttributePresent check as part of documentation-compliance Check tab summary:

IsInputAttributePresent check occurring in code in Files changed tab:

---

For reporting on cases like the null-handling, serialisation and versioning would now keep the information reporting only on the Checks tab

---

If that all makes sense? - interested in others thoughts!

[FraserGreenroyd]

For historical reference, @IsakNaslundBh and @pawelbaran were among those requesting annotation reports for versioning and serialisation checks (which was then extended automatically to null-handling on creation).

The reason for this was that it made a quick report at the top of the files changed tab of the state of versioning and serialisation on that pull request.

All 3 of the checks were, on original implementation, operating in the following fashion:

• Check out all repositories on the master or main branches as necessary for the pull request (for versioning this was all repos included in the installer, for the others just the repos needed to build that PR)
• Perform check, record the result
• Check out all repositories on the branch of the PR
• Perform check, record the result
• Compare results and highlight a failure if there are more or new errors on the PR than on master/main.

The annotation would then report to the PR so the human reviewer could make a judgement call as to whether the new errors were worth worrying about in that PR.

However, we have switched strategy slightly in recent weeks. Namely, for versioning we no longer compare to the master/main branches because there are no versioning errors on them, so any new errors should be squashed on being found. Similarly, the errors on master/main for null-handling are currently so high, they block out any meaningful changes from the PR.

I would therefore support a move for reports to be moved back to the reports tab, and off the annotations tab as you suggest @al-fisher 😄