Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hardcoded well-known sysconfigs for effortless cross compiling #896

Merged
merged 2 commits into from
May 4, 2022

Conversation

messense
Copy link
Member

@messense messense commented May 4, 2022

Implements #756 (comment)

With this when cross compiling for non-abi3 Linux wheels, there is no need to supply PYO3_CROSS_LIB_DIR anymore and -i pythonX.Y will just work™️. Internally maturin generates a pyo3 config file based on collected sysconfigs and set PYO3_CONFIG_FILE env var for pyo3's build script.

In theory we can extend the support to most Unix systems, for example macOS(added macOS in this PR generated from https://github.com/messense/python-sysconfig-collector), but that can be done in its dedicated PRs.

@netlify
Copy link

netlify bot commented May 4, 2022

Deploy Preview for maturin-guide ready!

Name Link
🔨 Latest commit 205b3dd
🔍 Latest deploy log https://app.netlify.com/sites/maturin-guide/deploys/62728492d9436f00080e3ad6
😎 Deploy Preview https://deploy-preview-896--maturin-guide.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@messense messense requested review from konstin and davidhewitt May 4, 2022 04:48
Copy link
Member

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a nice idea to me!

version={major}.{minor}
shared=true
abi3=false
build_flags=WITH_THREAD
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nb I think this flag only necessary on Python older than 3.7

src/build_options.rs Outdated Show resolved Hide resolved
}
}

let mut interpreters = Vec::new();
if binding_name.starts_with("pyo3") && target.is_unix() && target.cross_compiling() {
if let Some(cross_lib_dir) = std::env::var_os("PYO3_CROSS_LIB_DIR") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there also be a check for PYO3_CONFIG_FILE env var, and just skip all this if so?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that'd be nice, but it's out of scope of this PR, I'll open a new issue to track it.

@messense messense force-pushed the well-known-sysconfig branch from 7566a0d to 5f67d9a Compare May 4, 2022 06:58
@messense messense changed the title Add hard coded well-known sysconfigs for easier cross compiling Add hard coded well-known sysconfigs for effortless cross compiling May 4, 2022
@messense messense marked this pull request as ready for review May 4, 2022 07:26
@messense messense changed the title Add hard coded well-known sysconfigs for effortless cross compiling Add hardcoded well-known sysconfigs for effortless cross compiling May 4, 2022
@messense messense force-pushed the well-known-sysconfig branch from 5f67d9a to 205b3dd Compare May 4, 2022 13:50
Copy link
Member

@konstin konstin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH i don't understand much about pyo3's current cross compiling logic, but the code looks good.

Does this mean that we can now compile and cross-compile to a specific non-abi3 version on linux and mac without having that version installed?

@messense
Copy link
Member Author

messense commented May 4, 2022

Does this mean that we can now compile and cross-compile to a specific non-abi3 version on linux and mac without having that version installed?

Yes, that's the goal of this.

@messense messense merged commit 1b3316a into PyO3:main May 4, 2022
@messense messense deleted the well-known-sysconfig branch May 4, 2022 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants