[ci] automatically cancel GitHub Actions runs for outdated commits

[jameslamb]

Proposes modifying the GitHub Actions jobs that run on every commit, such that GitHub automatically cancels in-progress runs on a given branch when a new commit is pushed to that branch.

Benefits of this change

Once a new commit has been pushed to master or to a pull request, it's not important that CI jobs for the previous commit run to completion.

Automatically cancelling those in-progress CI runs that are going to be ignored anyway is helpful because it:

• increases the number of PRs that can be merged in a day
• reduces the impact that one very active PR can have on throughput for the rest of the repo
• reduces the load we place on GitHub Actions and other services the jobs hit
  • reduces the risk of triggering 409 - too many requests types of errors from services like GitHub, PyPI, CRAN, etc.
  • makes LightGBM a slightly better open source citizen (GitHub Actions is free and this project benefits from that)

It's important to remember that:

• for everything except the CUDA jobs, we're limited to 20 concurrent jobs across all workflows and commits (docs link)
• for the CUDA jobs, we're limited to 1 concurrent job across all workflows and commits, and we run 5 of those jobs on every commit

Notes for Reviewers

I first learned about that via this change to hypothesis: HypothesisWorks/hypothesis@b1f7a7c

See https://docs.github.com/en/actions/learn-github-actions/contexts for an explanation of the values that ${{ github.ref }} will take.

for branches the format is refs/heads/<branch_name>, for pull requests it is refs/pull/<pr_number>/merge, and for tags it is refs/tags/<tag_name>

[jmoralez]

Nice! I know this will definitely help with those annoying linter errors.

[jameslamb]

Yeah! And especially with the buildup of CUDA jobs when more than one person is active on the repo at the same time.

thanks to @Zac-HD for teaching me about this during the SciPy 2022 sprints 😊

[jameslamb]

Looks like this is working! I just merged a PR to master while another one's CUDA jobs were still running, and look what happened automatically:

Canceling since a higher priority waiting request for 'CUDA Version-refs/heads/master' exists

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.