Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: to_sql with ArrowExtesionArray #52058

Merged
merged 17 commits into from
Mar 22, 2023
Merged

BUG: to_sql with ArrowExtesionArray #52058

merged 17 commits into from
Mar 22, 2023

Conversation

mroeschke
Copy link
Member

Includes a fix for an oversight referenced in apache/arrow#34614

@mroeschke mroeschke added IO SQL to_sql, read_sql, read_sql_query Arrow pyarrow functionality labels Mar 17, 2023
@mroeschke mroeschke added this to the 2.0 milestone Mar 17, 2023
jbrockmendel
jbrockmendel reviewed Mar 17, 2023
View reviewed changes
pandas/io/sql.py Outdated
elif ser.dtype.kind == "m":
vals = ser._values
if isinstance(vals, ArrowExtensionArray):
vals = vals.to_numpy(dtype=np.dtype("m8"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can the dtype be more specific e.g. "m8[ns]"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. We warn that timedeltas will be stored as int with ns precision.

mroeschke added 13 commits March 17, 2023 15:23
@mroeschke
Copy link
Member Author

Some of the build timeout should be addressed with #52107

jbrockmendel
jbrockmendel approved these changes Mar 22, 2023
View reviewed changes
Copy link
Member

@jbrockmendel jbrockmendel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mroeschke mroeschke merged commit 3083ae9 into pandas-dev:main Mar 22, 2023
@mroeschke mroeschke deleted the bug/arrow/to_sql branch March 22, 2023 20:19
@lumberbot-app
Copy link

lumberbot-app bot commented Mar 22, 2023

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 2.0.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 3083ae932e0b122e963608fb25cf3df06d588b03
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #52058: BUG: to_sql with ArrowExtesionArray'
  1. Push to a named branch:
git push YOURFORK 2.0.x:auto-backport-of-pr-52058-on-2.0.x
  1. Create a PR against branch 2.0.x, I would have named this PR:

"Backport PR #52058 on branch 2.0.x (BUG: to_sql with ArrowExtesionArray)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

mroeschke added a commit to mroeschke/pandas that referenced this pull request Mar 22, 2023
@mroeschke
Backport PR pandas-dev#52058: BUG: to_sql with ArrowExtesionArray
376ec4c
mroeschke added a commit that referenced this pull request Mar 22, 2023
@mroeschke
Backport PR #52058 on branch 2.0.x (BUG: to_sql with ArrowExtesionArr…
7d5d123 
…ay) (#52124)

* Backport PR #52058: BUG: to_sql with ArrowExtesionArray

* _data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbrockmendel jbrockmendel jbrockmendel approved these changes

Assignees
No one assigned
Labels
Arrow pyarrow functionality IO SQL to_sql, read_sql, read_sql_query
Projects
None yet
Milestone
2.0
2 participants
@mroeschke @jbrockmendel