-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-43845: [Docs] Update Python Development Build Instructions #43617
base: main
Are you sure you want to change the base?
Conversation
@jorisvandenbossche @assignUser if you have a chance to look - might be something easy I am missing |
5cbc0d8
to
4965f74
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for looking into this. Updating those instructions was long overdue ..
@@ -398,7 +398,7 @@ Now, build pyarrow: | |||
|
|||
$ pushd arrow/python | |||
$ export PYARROW_PARALLEL=4 | |||
$ python setup.py build_ext --inplace | |||
$ python -m pip install -e . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ python -m pip install -e . | |
$ python -m pip install -e . --no-build-isolation --no-deps -v |
Those additional flags make it more similar to what we did before. It's probably debatable whether those should be added (disabling build isolation and installing deps), but since in this tutorial we have set up all build dependencies in advance, not doing that gives unnecessary pip overhead.
(this is also the command that I use locally, although for quick rebuilds I still use python setup.py build_ext --inplace
)
.. note:: | ||
To install an editable PyArrow build run ``pip install -e . --no-build-isolation`` | ||
in the ``arrow/python`` directory. | ||
$ python -m pip wheel . \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ python -m pip wheel . \ | |
$ python -m build . \ |
Using https://build.pypa.io/en/stable/ might be the recommended way nowadays to build a wheel?
When using that, it might need --config-setting
instead of --global-option
(but haven't verified this works the same)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When using that, it might need
--config-setting
instead of--global-option
(but haven't verified this works the same)
I believe the --global-option
is a requirement as long as setuptools is the backend. It's extremely murky, but here's the best conversation I could find on that topic:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we need some way to pass down the option, but what I meant is that when using python -m build
instead of python -m pip wheel
, the name of the keyword to pass options might be different.
Because in the docs of build
, I only see --config-setting
and not --global-option
@@ -548,7 +545,7 @@ Now, we can build pyarrow: | |||
|
|||
$ pushd arrow\python | |||
$ set CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1 | |||
$ python setup.py build_ext --inplace | |||
$ python pip install -e . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ python pip install -e . | |
$ python pip install -e . --no-build-isolation --no-deps -v |
(depending on what is done above)
|
(opened an issue for this, because it's not actually a minor change anymore) |
Rationale for this change
Following along the documentation as is will yield the errors described in these issues:
#42006 (comment)
#41992 (comment)
What changes are included in this PR?
The documentation now uses the pip front end, rather than invoking setuptools directly
Are these changes tested?
Locally
Are there any user-facing changes?
Yes...unfortunately when following these instructions, users will now see:
The pyarrow setup script seems to need --build-type and --bundle-arrow-cpp as global options. Unfortunately, there is a disconnect between what pip offers and what setuptools can handle, so the only current way to pass these options to setuptools is through the front-end that pip has deprecated. A larger discussion and recap of the issue can be seen here:
pypa/pip#11859 (comment)