BUG: DataFrame construction with dictionary ArrowDtype columns #53654
Conversation
mroeschke
added
Constructors
| import pyarrow as pa | ||
|
|
||
| if pa.types.is_dictionary(dtype.pyarrow_dtype): | ||
| other = other.astype(ArrowDtype(dtype.pyarrow_dtype.value_type)) |
There was a problem hiding this comment.
this is going to have implications beyond the constructors
Can you add test(s) that directly test the affected indexing methods
There was a problem hiding this comment.
Added some tests for get_indexer, get_indexer_non_unique that seem related. LMK if there are other methods you also had in mind
There was a problem hiding this comment.
new tests look good, thanks.
I think we can make this a lot cheaper though since the places that all _unpack_nested_dtype only need the result's dtype. so with a little tinkering we can just return that and avoid doing a cast
There was a problem hiding this comment.
Makes sense. Might be better as a followup since this change is being backported?
|
Looks to be greenish so merging, but can follow up if needed |
|
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free to suggest an improvement. |
…nary ArrowDtype columns
…s-dev#53654) * BUG: DataFrame construction with dictionary ArrowDtype columns * Add tests for get_indexer * Windows
…s-dev#53654) * BUG: DataFrame construction with dictionary ArrowDtype columns * Add tests for get_indexer * Windows
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.