Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pip packaging workflow to GHA #6938

Merged
merged 19 commits into from
Oct 13, 2022
Merged

Add pip packaging workflow to GHA #6938

merged 19 commits into from
Oct 13, 2022

Conversation

alexreinking
Copy link
Member

@alexreinking alexreinking commented Aug 15, 2022 •
edited
Loading

tldr:

$ pip install halide

This PR adds automated Python packaging to the Halide repo. It runs:

  • When a PR is merged, this workflow will run on main and push to the test PyPI index. This gives us sort-of nightlies for free.
  • When a release is tagged, this workflow will run on the tagged commit and push to the official PyPI index.

As this workflow is very expensive (Linux aarch64 takes several hours to complete) I don't want to spend our CI minutes on pull requests. We can fix Python packaging asynchronously until we find a cross-compiling solution that works for Linux aarch64.

In service of this, a few other changes had to be made:

  • cibuildwheel is a semi-official helper action for building a wheel against multiple versions of Python. It assumes that setup.py and supporting files are in the root of your repo. Rather than reinvent the wheel, I moved the files. C'est la vie.
  • The Adams2019 autoscheduler includes a hack for building "fat" (multiarch) objects on macOS. Our generators should probably learn that mac-universal2-64 is a thing.
  • I removed the code that detects the version number of the bindings because it was janky and breaking while I tried to test this. Maybe it can be restored one day so that our CMakeLists.txt (or perhaps some other file) can be the single source of truth for our version number. Alternatively, we could use something like bumpversion to make this process easier, but still manual.

@alexreinking alexreinking added buildbot_test_nothing Buildbots ignore this PR entirely. Takes precedence over all other buildbot_test_* labels. skip_buildbots Synonym for buildbot_test_nothing labels Aug 15, 2022
@alexreinking
Copy link
Member Author

alexreinking commented Aug 20, 2022 •
edited
Loading

Getting undiagnosable permissions errors... asking on GitHub Actions Discussions here: https://github.com/orgs/community/discussions/30173

Resolved by adding individual permissions to the halide/docker-images repository.

@alexreinking alexreinking marked this pull request as ready for review October 12, 2022 12:23
@alexreinking alexreinking merged commit 23e22cc into main Oct 13, 2022
ardier pushed a commit to ardier/Halide-mutation that referenced this pull request Mar 3, 2024
@alexreinking @ardier
Add pip packaging workflow to GHA (halide#6938)
e91c51b 
* Add pip packaging workflow

* Try adding steps for Windows/macOS

* Add CMake/Ninja to runners.

* Use MSVC on Windows

* Try caching LLVM build

* Split LLVM build into separate job

* Fix CI

* Fix CI

* Debugging

* Fix path?

* Tar for faster transfer and correct perms

* Don't build Halide as Universal 2

* Try using ClangCL on Windows

* hack in support for universal2 builds

* go back to universal2

* add universal2 to CIBW_ARCHS_MACOS

* Update workflow to publish to PyPI

* Add package metadata

* Last fixup before merge
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@steven-johnson steven-johnson steven-johnson approved these changes

Assignees
No one assigned
Labels
buildbot_test_nothing Buildbots ignore this PR entirely. Takes precedence over all other buildbot_test_* labels. skip_buildbots Synonym for buildbot_test_nothing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@alexreinking @steven-johnson