-
Notifications
You must be signed in to change notification settings - Fork 306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: converting to dataframe with out of bounds timestamps #209
Conversation
@tswast There is inconsistency with the existing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Regarding date_as_object
, it's a little different in that case, because it doesn't throw an error for dates. They just come back as strings if it's not set.
If we do provide timestamp_as_object
, I think it needs to be 3 states:
- (default) the behavior in this fix
- (explicitly false) let the error happen, since they want to use native pandas Timestamp (maybe for performance reasons)
- (explicitly true) always convert to datetime objects
Let me know if I should also add an explicit |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
I think we can wait for a separate PR for the timestamp_as_object
parameter feature.
Fixes #168.
This PR fixes the problem when converting query results to Pandas with
pyarrow
when data contains timestamps that would fall out ofpyarrow
's nanoseconds precision.The fix requires
pyarrow>=1.0.0
, thus it only works on Python 3.PR checklist