-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python] Improve the pyarrow build system #36411
Comments
Thanks @jorisvandenbossche ! |
Scipy has pioneered meson as the backend (in the wider python ecosystem at least), which is now being used by numpy & pandas as well, and is being picked up by more and more projects. For a regular python project it's probably a better choice than CMake-based builds (the syntax is already nice and pythonic too), though if you have substantial CMake infrastructure then that integrates more easily with scikit-build. |
We do have that, yes: https://github.com/apache/arrow/blob/main/python/CMakeLists.txt |
I'm aware, I've been building arrow for a while now. ;-) |
Also relevant issue: #36410 |
Major wish-list item: proper incremental / cached cython builds. |
I'll note that if |
So, caching should be usage for us in the next release of Cython: cython/cython#6070 (comment) |
The current build system of pyarrow consists of using setuptools with a custom
setup.py
that calls cmake to do the actual build. While this works, there are nowadays newer python "build backends" that we could benefit from to make this easier to maintain and use.Specifically build backends that are meant to work with CMake projects, such as
scikit-build
, might be worth exploring.The text was updated successfully, but these errors were encountered: