GH-38607: [Python] Disable PyExtensionType autoload #38608
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PyExtensionType autoload is really a misfeature. Disable it and deprecate PyExtensionType instantiation.
|
@raulcd You may want to pick this up once CI is green. |
github-actions
bot
added
Component: Python
Component: Documentation
awaiting review
|
@github-actions crossbow submit -g python -g wheel |
|
Revision: 1990150 Submitted crossbow builds: ursacomputing/crossbow @ actions-58962cba56
|
|
The dask, pandas, substrait, hdfs failures are existing nightly failures on main. |
raulcd
approved these changes
Nov 6, 2023
github-actions
bot
added
awaiting merge
raulcd
pushed a commit
that referenced
this pull request
Nov 6, 2023
### Rationale for this change PyExtensionType autoload is really a misfeature. It creates PyArrow-specific extension types, though using ExtensionType is almost the same complexity while allowing deserialization from non-PyArrow software. ### What changes are included in this PR? * Disable PyExtensionType autoloading and deprecate PyExtensionType instantiation. * Update the docs to emphasize ExtensionType. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * Closes: #38607 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
|
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit c73cb13. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them. |
8 tasks
JerAguilon
pushed a commit
to JerAguilon/arrow
that referenced
this pull request
Nov 9, 2023
### Rationale for this change PyExtensionType autoload is really a misfeature. It creates PyArrow-specific extension types, though using ExtensionType is almost the same complexity while allowing deserialization from non-PyArrow software. ### What changes are included in this PR? * Disable PyExtensionType autoloading and deprecate PyExtensionType instantiation. * Update the docs to emphasize ExtensionType. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * Closes: apache#38607 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this pull request
Nov 13, 2023
### Rationale for this change PyExtensionType autoload is really a misfeature. It creates PyArrow-specific extension types, though using ExtensionType is almost the same complexity while allowing deserialization from non-PyArrow software. ### What changes are included in this PR? * Disable PyExtensionType autoloading and deprecate PyExtensionType instantiation. * Update the docs to emphasize ExtensionType. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * Closes: apache#38607 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
|
Would it also be possible to patch this into version 12.0.1 or other versions that are compatible with python 3.7? I am stuck on a particular version and cannot patch. |
|
@milovanderlinden We have published a hotfix specifically for this: |
bveeramani
added a commit
to ray-project/ray
that referenced
this pull request
Nov 20, 2023
Arrow nightly tests are failing because our custom tensor extension uses pyarrow.PyExtensionType, and apache/arrow#38608 deprecated pyarrow.PyExtensionType in favor of pyarrow.ExtensionType. This PR updates our tensor extension to use the now-recommended APIs. --------- Signed-off-by: Balaji Veeramani <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Nov 29, 2023
Arrow nightly tests are failing because our custom tensor extension uses pyarrow.PyExtensionType, and apache/arrow#38608 deprecated pyarrow.PyExtensionType in favor of pyarrow.ExtensionType. This PR updates our tensor extension to use the now-recommended APIs. --------- Signed-off-by: Balaji Veeramani <[email protected]>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this pull request
Feb 19, 2024
### Rationale for this change PyExtensionType autoload is really a misfeature. It creates PyArrow-specific extension types, though using ExtensionType is almost the same complexity while allowing deserialization from non-PyArrow software. ### What changes are included in this PR? * Disable PyExtensionType autoloading and deprecate PyExtensionType instantiation. * Update the docs to emphasize ExtensionType. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * Closes: apache#38607 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
PyExtensionType autoload is really a misfeature. It creates PyArrow-specific extension types, though using ExtensionType is almost the same complexity while allowing deserialization from non-PyArrow software.
What changes are included in this PR?
Are these changes tested?
Yes.
Are there any user-facing changes?
Yes.