Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimum required pyarrow version #1174

Closed
gli-chris-hao opened this issue Sep 13, 2024 · 2 comments
Closed

Minimum required pyarrow version #1174

gli-chris-hao opened this issue Sep 13, 2024 · 2 comments

Comments

@gli-chris-hao
Copy link

Apache Iceberg version

0.7.1 (latest release)

Please describe the bug 🐞

  1. pyiceberg calls pyarrow concat_tables with promote_options here: https://github.com/apache/iceberg-python/blob/pyiceberg-0.7.1/pyiceberg/io/pyarrow.py#L1353

  2. promote_options is only available in and after pyarrow 14.0.0 (added in this PR: GH-36845: [C++][Python] Allow type promotion on pa.concat_tables arrow#36846)

  3. however pyarrow version is pinned to ">=9.0.0,<18.0.0" in https://github.com/apache/iceberg-python/blob/pyiceberg-0.7.1/pyproject.toml#L61, I think it should be updated to ">=14.0.0,<18.0.0"

@kevinjqliu
Copy link
Contributor

Thanks for reporting this. That makes sense to me. Is this something you would like to help contribute?

@sungwy
Copy link
Collaborator

sungwy commented Sep 17, 2024

Hi @gli-chris-hao thank you for reporting this issue! @ndrluis has already made a contribution to fix this in main

@sungwy sungwy closed this as completed Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants