Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make casting from a timestamp without timezone to a timestamp with timezone configurable #6555

Closed
niebayes opened this issue Oct 14, 2024 · 1 comment
Labels
enhancement Any new improvement worthy of a entry in the changelog

Comments

@niebayes
Copy link
Contributor

niebayes commented Oct 14, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

In our time-series database project, we always interpret timestamps without a timezone as being in the UTC timezone. However, when we use the arrow cast function to convert a timestamp without a timezone to a specific timezone, the result isn't what we expect. This is because the cast function enforces an adjustment internally, which causes an unintended modification to the timestamp values during the cast process.

As specified in the doc and in the comment of codes, arrow cast will adjust the timestamp values if casting from a timestamp array without a timezone to a timestamp array with a timezone.

I propose to make such a behavior configurable and optional.

Describe the solution you'd like

A sound solution may be adding a boolean flag to CastOptions to make the arrow cast function not perform such an adjustment. This boolean flag may be named enable_timestamp_adjustment.

Describe alternatives you've considered

Additional context

#5827 stated that the behavior of the arrow cast function is somewhat confusing.

@niebayes niebayes added the enhancement Any new improvement worthy of a entry in the changelog label Oct 14, 2024
@niebayes niebayes changed the title Make casting from a timestamp without timezone to a timestamp with timezone optional Make casting from a timestamp without timezone to a timestamp with timezone configurable Oct 14, 2024
@tustvold
Copy link
Contributor

tustvold commented Oct 14, 2024

Whilst perhaps not ideal, this behaviour can be achieved by first casting to a timestamp with a timezone of UTC and then casting to the desired target TZ.

That being said, if all timestamps are UTC, why not just store them as UTC timestamps? It isn't just the cast kernels that would need to be updated to support a non-spec-compliant interpretation of timestamps without a timezone.

@tustvold tustvold closed this as not planned Won't fix, can't repro, duplicate, stale Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

No branches or pull requests

2 participants