Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Add GHA workflow for minimal CI jobs #750

Merged
merged 43 commits into from
Jan 19, 2023

Conversation

atolopko-czi
Copy link
Member

@atolopko-czi atolopko-czi commented Jan 19, 2023

Issue and/or context:

#723

Changes:

  • add python-ci-minimal.yml GHA workflow for minimal CI jobs, invoked by pushes to feature branches
  • rename python-ci.yml to python-ci-full.yml, to differentiate from the "minimal" workflow; invoked by pushes to main and for releases
  • add python-ci-single.yml, which is a reusable workflow called by the "minimal" and "full" workflows.
  • factored out linting steps into a separate job from the build/tests steps to avoid having conditional logic on those steps

Notes for Reviewer:

Successful workflow runs:

This will run only for pull requests
@atolopko-czi atolopko-czi self-assigned this Jan 19, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jan 19, 2023

Codecov Report

Base: 78.76% // Head: 79.35% // Increases project coverage by +0.58% 🎉

Coverage data is based on head (5e2689d) compared to base (762fd68).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #750      +/-   ##
==========================================
+ Coverage   78.76%   79.35%   +0.58%     
==========================================
  Files          30       30              
  Lines        1978     1981       +3     
==========================================
+ Hits         1558     1572      +14     
+ Misses        420      409      -11     
Flag Coverage Δ
python 79.35% <ø> (+0.58%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
apis/python/src/tiledbsoma/collection.py 86.66% <0.00%> (-1.67%) ⬇️
apis/python/src/tiledbsoma/sparse_nd_array.py 94.06% <0.00%> (-0.85%) ⬇️
apis/python/src/tiledbsoma/dense_nd_array.py 93.10% <0.00%> (+0.08%) ⬆️
apis/python/src/tiledbsoma/util_arrow.py 92.85% <0.00%> (+0.12%) ⬆️
apis/python/src/tiledbsoma/query_condition.py 61.05% <0.00%> (+0.18%) ⬆️
apis/python/src/tiledbsoma/io.py 84.50% <0.00%> (+0.28%) ⬆️
apis/python/src/tiledbsoma/factory.py 89.79% <0.00%> (+2.04%) ⬆️
apis/python/src/tiledbsoma/eta.py 84.21% <0.00%> (+2.63%) ⬆️
apis/python/src/tiledbsoma/util_ann.py 15.56% <0.00%> (+4.79%) ⬆️
apis/python/src/tiledbsoma/logging.py 80.00% <0.00%> (+20.00%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@johnkerl johnkerl changed the title add GHA workflow for minimal CI jobs [python] Add GHA workflow for minimal CI jobs Jan 19, 2023
@atolopko-czi atolopko-czi requested a review from johnkerl January 19, 2023 18:59
@atolopko-czi atolopko-czi marked this pull request as ready for review January 19, 2023 19:38
already run my pre-commit; erroneously left in after merging from main
name: TileDB-SOMA Python CI (Minimal)

on:
push:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now differs from r-ci.yml, which is not ideal. And the GHA console shows push- and pull_request-triggered workflow runs somewhat differently. I'm going to test once more with pull_request...

@johnkerl johnkerl self-requested a review January 19, 2023 20:32
@atolopko-czi
Copy link
Member Author

atolopko-czi commented Jan 19, 2023

@johnkerl @thetorpedodog I'd say this is ready, if we want to move forward with it. I merged in the caching improvements that Paul made earlier today.

I never quite figured out why only push works to trigger Python workflows, while pull_request is working to trigger R workflows. Interestingly, the R workflow triggers even if just a apis/python/** file changes (it shouldn't), which makes me believe it's not the pull_request trigger that is working, but rather the push trigger that also exists in the r-ci.yml file. It must be that a transient merge into main is being done to test the PR against main, and that is what's triggering the R workflow. It's a bit mind-boggling and GHA docs are not clear on the matter at all. But they hint at such things as I've described...

Not going to spend more time on this, so we can take it or leave it as is. 😄

(I believe the R build failure is known and may be resolved with another merge from main.)

@johnkerl
Copy link
Member

Yes @atolopko-czi re known-broken R CI pending #766 :)

Copy link
Member

@johnkerl johnkerl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@atolopko-czi thank you thank you thank you 🙏

I know this was a long slog, and a puzzler, with lots of revs trying to untangle various complexities

But I really do apprecate it -- the more people are actively contributing to SOMA right now, the more we're using CI -- and your work here will help us _all_move faster. :D

@johnkerl johnkerl merged commit 190f216 into main Jan 19, 2023
@johnkerl johnkerl deleted the atol/693-ci-optimize-resource-usage branch January 19, 2023 22:51
@johnkerl
Copy link
Member

See also #693

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants