You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In our time-series database project, we always interpret timestamps without a timezone as being in the UTC timezone. However, when we use the arrow cast function to convert a timestamp without a timezone to a specific timezone, the result isn't what we expect. This is because the cast function enforces an adjustment internally, which causes an unintended modification to the timestamp values during the cast process.
As specified in the doc and in the comment of codes, arrow cast will adjust the timestamp values if casting from a timestamp array without a timezone to a timestamp array with a timezone.
I propose to make such a behavior configurable and optional.
Describe the solution you'd like
A sound solution may be adding a boolean flag to CastOptions to make the arrow cast function not perform such an adjustment. This boolean flag may be named enable_timestamp_adjustment.
Describe alternatives you've considered
Additional context
#5827 stated that the behavior of the arrow cast function is somewhat confusing.
The text was updated successfully, but these errors were encountered:
niebayes
changed the title
Make casting from a timestamp without timezone to a timestamp with timezone optional
Make casting from a timestamp without timezone to a timestamp with timezone configurable
Oct 14, 2024
Whilst perhaps not ideal, this behaviour can be achieved by first casting to a timestamp with a timezone of UTC and then casting to the desired target TZ.
That being said, if all timestamps are UTC, why not just store them as UTC timestamps? It isn't just the cast kernels that would need to be updated to support a non-spec-compliant interpretation of timestamps without a timezone.
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In our time-series database project, we always interpret timestamps without a timezone as being in the UTC timezone. However, when we use the arrow cast function to convert a timestamp without a timezone to a specific timezone, the result isn't what we expect. This is because the cast function enforces an adjustment internally, which causes an unintended modification to the timestamp values during the cast process.
As specified in the doc and in the comment of codes, arrow cast will adjust the timestamp values if casting from a timestamp array without a timezone to a timestamp array with a timezone.
I propose to make such a behavior configurable and optional.
Describe the solution you'd like
A sound solution may be adding a boolean flag to
CastOptions
to make the arrow cast function not perform such an adjustment. This boolean flag may be namedenable_timestamp_adjustment
.Describe alternatives you've considered
Additional context
#5827 stated that the behavior of the arrow cast function is somewhat confusing.
The text was updated successfully, but these errors were encountered: