Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the HWSAPI bot functionalities through GitHub Actions #3144

Merged

Conversation

MishimaHaruna
Copy link
Member

Pull Request Prelude

Changes Proposed

This reimplements the "HWSAPI" build bot functionalities as a GitHub Actions workflow.

The HWSAPI build bot currently has the following duties (on each push to master):

  • Updates the HPMHooking plugin
  • Updates the SQL item DB script
  • Updates the SQL mob/mob skill DB script
  • Updates the constants documentation
  • Pushes the changes to master, if any
  • Publishes the build time to https://herc.ws/hwsapi/buildstats

The current buildbot is implemented as part of the legacy IRC bot (on a modified map-server core), which is scheduled to be retired within June 16th 2022 as part of an ongoing infrastructure modernization process and the discontinuation of the old hardware that was running it.

This action relies on two github secrets (GPG key and buildstats API key), which have already been added to the main repository.

An example of execution of this script can be seen here: https://github.com/MishimaHaruna/Hercules/runs/6639710150

A drawback of this approach is that (apparently) the order in which GitHub Actions workflows are executed isn't deterministic, as they're all started in parallel and handed over to runners as they become available. If anyone has any idea on how to prioritize this one so that it's scheduled before the other workflows (static analysis, tests), so that the changes are pushed as soon as possible, please suggest it.

@MishimaHaruna MishimaHaruna added this to the Release v2022.06.01 milestone May 29, 2022
@MrKeiKun
Copy link
Contributor

@MishimaHaruna i think the parallel job can be solved using concurrency

@4144
Copy link
Contributor

4144 commented May 29, 2022

@MishimaHaruna
Copy link
Member Author

I think concurrency will help, but won't fully solve the problem, as it only ensures that jobs (or workflows) with the same concurrency group are never executed in parallel, but it doesn't give control over the order (i.e. I can't put everything in the sam concurrency group to execute them serially, since I still have no way to tell it that the HWSAPI one should be executed first). It seems very useful in order to cancel previous executions of th HWSAPI build if subsequent commits are pushed though, since it'd fail to push its changes anyway.

The stack overflow suggestion on how to implement some scheduling order between workflows looks interesting, I believe I can try and build on it. It's quite unfortunate that they don't support something like that natively without having to rely on that hack with workflows triggering each others but it seems feasible

@MishimaHaruna MishimaHaruna changed the title Implement the HWSAPI bot functionalities through GitHub Actions WIP: Implement the HWSAPI bot functionalities through GitHub Actions May 30, 2022
@MishimaHaruna MishimaHaruna marked this pull request as draft May 30, 2022 15:52
@MishimaHaruna MishimaHaruna changed the title WIP: Implement the HWSAPI bot functionalities through GitHub Actions Implement the HWSAPI bot functionalities through GitHub Actions May 30, 2022
@MishimaHaruna MishimaHaruna force-pushed the hwsapi-github-actions branch 5 times, most recently from efd0e29 to 0b5dabb Compare May 30, 2022 22:46
@MishimaHaruna
Copy link
Member Author

I believe I reached a satisfactory compromise. The workflows are executes as follows:

For pull requests:

  • Each workflow is executed independently, through its on: pull_request directive, exactly as before.
  • The hwsapibot is not executed

For pushes:

  • Only the controller workflow is executed, through its on: push directive. The controller:
    • If the branch is master, runs the hwsapibot workflow, otherwise it doesn't
    • If the hwsapibot workflow was successful (or skipped because not on the master branch), runs both the codeql-analysis and the tools workflows in parallel
    • If the tools workflow was successful (this is to quickly detect any issues in the SQL files without having to wait for the entire build matrix to run), it runs the build workflow
    • If the build workflow was successful (again, to detect failures on the most common build settings), it launches all the remaining workflows in parallel
  • The hwsapibot workflow will cancel its previous instances if a new execution (i.e. a new push to master) happens before it completes.

The conditions in the controller workflow are a little weird looking, but that seems to be the only way to make them continue if a job they depend on (either directly or indirectly) is skipped. This is documented in actions/runner#491

Here is an example of a run on the master branch (on my fork): https://github.com/MishimaHaruna/Hercules/actions/runs/2411704198
Here is an example of a run on a non-master branch (note the skipped hwsapibot/hwsapi job that doesn't cause the dependent jobs to be skipped or cancelled): https://github.com/MishimaHaruna/Hercules/actions/runs/2411706027

@MishimaHaruna MishimaHaruna marked this pull request as ready for review May 30, 2022 23:10
@MrKeiKun
Copy link
Contributor

@MishimaHaruna everything looks good. Btw, out of the PR but still can be added as separate commit, time to make CodeQL use v2 of it.

@MishimaHaruna
Copy link
Member Author

Good point. I checked their docs and I don't see any differences in basic usage, so it hopefully won't break anything

@MishimaHaruna MishimaHaruna requested a review from 4144 June 1, 2022 13:53
@MishimaHaruna MishimaHaruna merged commit 382bbdc into HerculesWS:master Jun 1, 2022
@MishimaHaruna MishimaHaruna deleted the hwsapi-github-actions branch June 1, 2022 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants