Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip pr-merge at pull request submission or update #84

Closed
gilles-peskine-arm opened this issue Dec 20, 2022 · 1 comment
Closed

Skip pr-merge at pull request submission or update #84

gilles-peskine-arm opened this issue Dec 20, 2022 · 1 comment
Labels
enhancement New feature or request

Comments

@gilles-peskine-arm
Copy link
Contributor

To reduce the load on the CI infrastructure, run pr-merge only nightly (only if either the target branch or the head branch has been updated recently, as is the case today). When a pull request is submitted or updated, only run pr-head.

Until today, and since 21 November 2022, we were only running pr-merge (on PR submission/update and nightly), never pr-head. However the absence of pr-head has some downsides which made it tolerable as a temporary workaround, but not acceptable forever:

  • When a pull request has merge conflict, no CI runs.
  • We only get CI results for the merge, which can sometimes be hard to trace to the pre-merge code. This is mostly a problem for CI results that are hard to reproduce locally (non-Linux platforms, non-free compilers).
  • When a pull request hasn't been updated for a while, the only status visible on GitHub is “did not run”. So when looking for things to review or gatekeep, we can't distinguish “this needs work before it'll pass the CI” from “this passed the CI against a slightly older version of the target branch, so it's probably ok”. Digging in the previous builds on Jenkins might give the information, but it's tedious to figure out if there was a success on the latest head commit, and the history is erased after a while.

Today 20 December 2022, I have re-enabled pr-head. This is necessary for Mbed-TLS/mbedtls#6359, which will happen early in January 2023 and will lead to a temporary increased need for running the CI on pull requests with merge conflicts.

@gilles-peskine-arm gilles-peskine-arm added the enhancement New feature or request label Dec 20, 2022
@gilles-peskine-arm
Copy link
Contributor Author

The core issue of CI overload is resolved by #91 and #117:

  • We now run pr-head, with a full CI campaign (as defined by our pull request job: all.sh + Windows testing + result analysis).
  • Disable all jobs except API-ABI check in PR-merge #91 more than halves the CI load by reducing pr-merge to the interface stability check.
  • Enable merge queues on the mbedtls repo #117 restores and improves on testing the pull request merged with its target branch: we now use a merge queue, so the CI runs on the actual merge commit before the merge becomes official. This is an improvement on the old pr-merge situation because that only ran nightly so the result could still be out of date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant