Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs #906

seisman · 2021-02-16T02:15:04Z

Description of proposed changes

This PR is inspired by another PR #869 from @willschlitzer.

This PR improves the "Tests" workflow to save CI resources.

Expected behavior (see the comments below for how it actually works):

For draft PRs: only run tests on Ubuntu + Python 3.9 (~10 minutes)
When draft PRs are ready for review, run all CI jobs (9 jobs, 10-30 minutes)
If converting PRs back to draft mode, pushes still only trigger one job (~10 minutes)
After merged into master, all 9 jobs will run.

References:

Closes #869.

This PR makes changes to save CI resources: - For draft PRs: only run tests on Ubuntu + Python 3.9 (~10 minutes) - When draft PRs are ready for review, run all CI jobs (9 jobs, 10-30 minutes) - If converting PRs back to draft mode, pushes still only trigger one job

seisman · 2021-02-16T02:29:08Z

Now, this PR is in draft mode. As expected, commit af67396 only triggers one job (Ubuntu + Python 3.9). Check https://github.com/GenericMappingTools/pygmt/actions/runs/570286139 for the job details. Below is the screenshot of the CI runs. Only the "ubuntu-latest - Python 3.9" job is triggered. Other jobs are waiting for status because they're not triggered.

seisman · 2021-02-16T02:32:17Z

This PR is still in draft mode. New commits like 7c6309d still only triggers one job (ubuntu-latest - Python 3.9). Check https://github.com/GenericMappingTools/pygmt/actions/runs/570313438 for job details. Here is the screenshot of the CI runs. Only one is running (in progress).

seisman · 2021-02-16T02:40:58Z

I just pushed the "Ready for Review" button, which triggers all the CI jobs. Check https://github.com/GenericMappingTools/pygmt/actions/runs/570319849 for job details.

Below is the screenshot for job status. Some notes:

All 9 jobs are triggered now, and the old "ubuntu-latest - Python 3.9" job is not canceled. So 10 jobs have been triggered by the commit and the "ready_for_review" event.
The "Style Checks" workflow is not re-triggered by "ready for review" event.
Three jobs in the "GMT Dev Tests" workflow are also triggered by the "ready_for_review" event.

seisman · 2021-02-16T02:54:17Z

The PR is not in draft mode, so new commits like 4dc7e87 trigger new CI runs (9 jobs). Check https://github.com/GenericMappingTools/pygmt/actions/runs/570349410 for job details.

Some notes:

The new commit triggers new CI runs (9 jobs) in the "Tests" workflow
This new commit does not trigger any runs in the "GMT Dev Tests" workflow, because the workflow is only triggered by the ready_for_review event.
Some of the old, unfinished jobs in https://github.com/GenericMappingTools/pygmt/actions/runs/570319849 are canceled.

seisman · 2021-02-16T03:06:32Z

PR authors and project maintainers can convert a PR into draft mode. Note that "convert to draft" is not an "event", so it doesn't trigger new CI runs.

seisman · 2021-02-16T03:14:01Z

Now the PR is in draft mode. New commits like 06d0073 only triggers one job (ubuntu-latest + Python 3.9). Check https://github.com/GenericMappingTools/pygmt/actions/runs/570386483 for job details.

seisman · 2021-02-16T03:19:54Z

Now I can make more changes in draft mode. Each commit only triggers one job, so saving us 8/9 CI resources, especially for macOS and Windows.

Like above, if I mark the PR as ready for review again, 9 jobs will be triggered.

So what will happen if the changes are merged into master branch? I can't show you in this repository. Instead, I made similar changes to my own fork (https://github.com/seisman/pygmt), and merge the changes into my master branch. Everything works well. See the screenshot below and https://github.com/seisman/pygmt/actions/runs/570241183 for job details.

The "Publish to PyPI" workflow fails as reported in #901.

seisman · 2021-02-16T03:27:49Z

Now this PR is ready for review.

weiji14

This is awesome, now I won't feel so bad burning through so many CPU cycles! Just a few comments.

MAINTENANCE.md

.github/workflows/ci_tests.yaml

Co-authored-by: Wei Ji <[email protected]>

weiji14

Up to you if you want to apply the changes below to close #908, otherwise good to merge.

weiji14 · 2021-02-16T04:58:20Z

MAINTENANCE.md

+In draft Pull Requests, only one job (Ubuntu + Python latest)
+is triggered to save on Continuous Integration resources.


Suggested change

In draft Pull Requests, only one job (Ubuntu + Python latest)

is triggered to save on Continuous Integration resources.

To save on Continuous Integration resources, keep your Pull Request

in draft mode so that only one job (Ubuntu + Python latest) is triggered.

weiji14 · 2021-02-16T05:00:05Z

MAINTENANCE.md

@@ -58,6 +58,8 @@ It is also scheduled to run daily on the *master* branch.

 This is ran on every commit to the *master* and Pull Request branches.
 It is also scheduled to run daily on the *master* branch.
+In draft Pull Requests, only one job (Ubuntu + Python latest)
+is triggered to save on Continuous Integration resources.



Suggested change

Alternatively, add [skip ci] to your commit message so that Github Actions won't run.

See https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/.

Prefer to leave it to another PR, as more tests need to be done before documenting it.

Ok, feel free to copy this text to that next PR.

…enericMappingTools#906) This PR improves the "Tests" workflow to save CI resources: - For draft PRs: only run tests on Ubuntu + Python 3.9 (~10 minutes) - When draft PRs are ready for review, run all CI jobs (9 jobs, 10-30 minutes) - If converting PRs back to draft mode, pushes still only trigger one job (~10 minutes) - After merged into master, all 9 jobs will run. * Update MAINTENANCE.md Co-authored-by: Wei Ji <[email protected]>

Commit changes in draft PR

7c6309d

vercel bot temporarily deployed to Preview February 16, 2021 02:29 Inactive

seisman marked this pull request as ready for review February 16, 2021 02:33

Commit changes to trigger CI jobs in ready-for-review PR

4dc7e87

vercel bot temporarily deployed to Preview February 16, 2021 02:50 Inactive

seisman marked this pull request as draft February 16, 2021 03:05

Update MAINTENANCE.md

06d0073

vercel bot temporarily deployed to Preview February 16, 2021 03:12 Inactive

seisman added this to the 0.4.0 milestone Feb 16, 2021

seisman added the maintenance Boring but important stuff for the core devs label Feb 16, 2021

seisman marked this pull request as ready for review February 16, 2021 03:27

seisman requested review from a team February 16, 2021 03:28

weiji14 reviewed Feb 16, 2021

View reviewed changes

MAINTENANCE.md Outdated Show resolved Hide resolved

.github/workflows/ci_tests.yaml Outdated Show resolved Hide resolved

.github/workflows/ci_tests.yaml Show resolved Hide resolved

weiji14 changed the title ~~CI: Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs~~ CI: Run tests on a single CI job (Ubuntu + Python 3.0) for draft PRs Feb 16, 2021

weiji14 changed the title ~~CI: Run tests on a single CI job (Ubuntu + Python 3.0) for draft PRs~~ CI: Run tests on a single CI job (Ubuntu + Python 3.9) for draft PRs Feb 16, 2021

weiji14 changed the title ~~CI: Run tests on a single CI job (Ubuntu + Python 3.9) for draft PRs~~ CI: Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs Feb 16, 2021

Apply suggestions from code review

d2e41fa

Co-authored-by: Wei Ji <[email protected]>

vercel bot temporarily deployed to Preview February 16, 2021 04:39 Inactive

weiji14 mentioned this pull request Feb 16, 2021

Document how to skip CI in the contributing guides #908

Closed

weiji14 approved these changes Feb 16, 2021

View reviewed changes

seisman merged commit 6fec148 into master Feb 16, 2021

seisman deleted the save-ci branch February 16, 2021 05:05

seisman mentioned this pull request Feb 16, 2021

Add test file for draft pull requests #869

Closed

5 tasks

seisman modified the milestones: 0.4.0, 0.3.1 Feb 16, 2021

seisman changed the title ~~CI: Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs~~ Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs Feb 17, 2021

weiji14 mentioned this pull request Mar 18, 2021

NEP29: Set minimum required versions to NumPy 1.17+ and Python 3.7+ #1074

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs #906

Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs #906

seisman commented Feb 16, 2021 •

edited

Loading

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021 •

edited

Loading

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021 •

edited

Loading

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021

weiji14 left a comment

weiji14 left a comment

weiji14 Feb 16, 2021

weiji14 Feb 16, 2021

seisman Feb 16, 2021

weiji14 Feb 16, 2021

		In draft Pull Requests, only one job (Ubuntu + Python latest)
		is triggered to save on Continuous Integration resources.


	Alternatively, add [skip ci] to your commit message so that Github Actions won't run.
	See https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/.

Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs #906

Run tests in a single CI job (Ubuntu + Python 3.9) for draft PRs #906

Conversation

seisman commented Feb 16, 2021 • edited Loading

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021 • edited Loading

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021 • edited Loading

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021

seisman commented Feb 16, 2021

weiji14 left a comment

Choose a reason for hiding this comment

weiji14 left a comment

Choose a reason for hiding this comment

weiji14 Feb 16, 2021

Choose a reason for hiding this comment

weiji14 Feb 16, 2021

Choose a reason for hiding this comment

seisman Feb 16, 2021

Choose a reason for hiding this comment

weiji14 Feb 16, 2021

Choose a reason for hiding this comment

seisman commented Feb 16, 2021 •

edited

Loading

seisman commented Feb 16, 2021 •

edited

Loading

seisman commented Feb 16, 2021 •

edited

Loading