Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thoughts on adding read_delta to pandas #869

Closed
MrPowers opened this issue Oct 9, 2022 · 2 comments
Closed

Thoughts on adding read_delta to pandas #869

MrPowers opened this issue Oct 9, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@MrPowers
Copy link
Collaborator

MrPowers commented Oct 9, 2022

There is an open pandas issue on Delta Lake file format support.

@houqp made this suggestion in the thread:

Perhaps we could add https://pypi.org/project/deltalake/ as an optional extra dependency to pandas itself to make deltalake support work out of the box for pandas users?

pandas already has read_csv, read_excel, read_parquet, etc., so read_delta would be a natural fit.

Some thoughts to brainstorm as a team:

  • would adding read_delta to pandas be a good idea? I certainly think so, but interested if there are any objections
  • Is read_delta a good method name or perhaps read_deltalake?
  • What method signature would we need? We'd at least need an optional version keyword argument for time travel 😎
@MrPowers MrPowers added the enhancement New feature or request label Oct 9, 2022
@fvaleye
Copy link
Collaborator

fvaleye commented Nov 16, 2022

A Pull Request has been submitted to integrate directly deltalake in the code base as an optional external dependency.

@fvaleye
Copy link
Collaborator

fvaleye commented Dec 1, 2022

Closing this, based on the response from pandas core devs in pandas-dev/pandas#49692

@fvaleye fvaleye closed this as completed Dec 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants