GH-30117: [C++][Python] Add "Z" to the end of timestamp print string when tz defined #39272

AlenkaF · 2023-12-18T09:29:45Z

What changes are included in this PR?

This PR updates the PrettyPrint for Timestamp type so that "Z" is printed at the end of the output string if the timezone has been defined. This way we add minimum information about the values being stored in UTC.

Are these changes tested?

Yes.

Are there any user-facing changes?

There is a change in how TimestampArray prints out the data. With this change "Z" would be added to the end of the string if the timezone is defined.

Closes: [C++][Python][R] PrettyPrint ignores timezone #30117

github-actions · 2023-12-18T09:30:14Z

⚠️ GitHub issue #30117 has been automatically assigned in GitHub to PR creator.

rok

Seems good. Let's just check that weird zones don't cause issues.

python/pyarrow/tests/test_types.py

cpp/src/arrow/util/formatting_util_test.cc

Co-authored-by: Rok Mihevc <[email protected]>

python/pyarrow/tests/test_types.py

Co-authored-by: Rok Mihevc <[email protected]>

rok · 2023-12-18T14:53:45Z

Forgot to ask: does this currently print timestamp's type when printing the array? And if it does can you add a test that it prints the correct timezone. If not we can leave it for the follow-up.

AlenkaF · 2023-12-18T15:02:39Z

Forgot to ask: does this currently print timestamp's type when printing the array? And if it does can you add a test that it prints the correct timezone. If not we can leave it for the follow-up.

No, when printing the array there is no info about the type, one should still inspect the type separately:

>>> import pyarrow as pa
>>> a = pa.array([0], pa.timestamp('s', tz='+02:00'))
>>> a
<pyarrow.lib.TimestampArray object at 0x131399fc0>
[
  1970-01-01 00:00:00Z
]

>>> a.type
TimestampType(timestamp[s, tz=+02:00])

as mentioned on the issue, we can add this if find important.

rok · 2023-12-18T15:34:23Z

as mentioned on the issue, we can add this if find important.

If there's an easy way to include timezone it would be great!

cpp/src/arrow/util/formatting_util_test.cc

jorisvandenbossche · 2023-12-19T08:54:58Z

does this currently print timestamp's type when printing the array?

Given this is a completely separate code path (this is something defined in the python repr, not the C++ pretty printer), let's keep that for a separate PR?
(but yes, I agree it would be good to do that!)

rok · 2023-12-19T09:50:48Z

Given this is a completely separate code path (this is something defined in the python repr, not the C++ pretty printer), let's keep that for a separate PR? (but yes, I agree it would be good to do that!)

Agreed, let's make another issue for it.

AlenkaF · 2023-12-19T10:42:45Z

Given this is a completely separate code path (this is something defined in the python repr, not the C++ pretty printer), let's keep that for a separate PR? (but yes, I agree it would be good to do that!)

Agreed, let's make another issue for it.

It would be good to add that to R also, if I am not mistaken. Will create an issue for it 👍

cpp/src/arrow/util/formatting_util_test.cc

AlenkaF · 2023-12-20T09:25:47Z

Opened an issue for printing timezone information in TimestampArray: #39315
cc @rok @jorisvandenbossche

jorisvandenbossche · 2024-01-08T13:26:31Z

Thanks!

conbench-apache-arrow · 2024-01-08T22:34:50Z

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit a288364.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them.

…tring when tz defined (apache#39272) ### What changes are included in this PR? This PR updates the PrettyPrint for Timestamp type so that "Z" is printed at the end of the output string if the timezone has been defined. This way we add minimum information about the values being stored in UTC. ### Are these changes tested? Yes. ### Are there any user-facing changes? There is a change in how `TimestampArray` prints out the data. With this change "Z" would be added to the end of the string if the timezone is defined. * Closes: apache#30117 Lead-authored-by: AlenkaF <[email protected]> Co-authored-by: Alenka Frim <[email protected]> Co-authored-by: Rok Mihevc <[email protected]> Signed-off-by: Joris Van den Bossche <[email protected]>

…' when formatting timestamps (#41045) ### What changes are included in this PR? A test that reproduces an issue found by the fuzzer and a fix for it. ### Are these changes tested? - A test - Comments clarifying somethings around `formatting.h` - Increasing the size of the local buffer used to format timestamps The issue was introduced only recently (unreleased): #39272 * GitHub Issue: #41044 Authored-by: Felipe Oliveira Carvalho <[email protected]> Signed-off-by: Felipe Oliveira Carvalho <[email protected]>

…the 'Z' when formatting timestamps (apache#41045) ### What changes are included in this PR? A test that reproduces an issue found by the fuzzer and a fix for it. ### Are these changes tested? - A test - Comments clarifying somethings around `formatting.h` - Increasing the size of the local buffer used to format timestamps The issue was introduced only recently (unreleased): apache#39272 * GitHub Issue: apache#41044 Authored-by: Felipe Oliveira Carvalho <[email protected]> Signed-off-by: Felipe Oliveira Carvalho <[email protected]>

Initial commit

4dd2ce1

github-actions bot added Component: C++ Component: Python awaiting review Awaiting review labels Dec 18, 2023

AlenkaF marked this pull request as ready for review December 18, 2023 10:36

rok reviewed Dec 18, 2023

View reviewed changes

python/pyarrow/tests/test_types.py Outdated Show resolved Hide resolved

cpp/src/arrow/util/formatting_util_test.cc Outdated Show resolved Hide resolved

github-actions bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Dec 18, 2023

Apply suggestions from code review - rok

98ec744

Co-authored-by: Rok Mihevc <[email protected]>

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 18, 2023

AlenkaF added 2 commits December 18, 2023 13:26

Run linter

9616771

Remove redundant line in formatting_util_test.cc

944b1c2

rok reviewed Dec 18, 2023

View reviewed changes

python/pyarrow/tests/test_types.py Outdated Show resolved Hide resolved

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Dec 18, 2023

Update python/pyarrow/tests/test_types.py

ca92eb7

Co-authored-by: Rok Mihevc <[email protected]>

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 18, 2023

rok approved these changes Dec 18, 2023

View reviewed changes

github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Dec 18, 2023

jorisvandenbossche reviewed Dec 19, 2023

View reviewed changes

cpp/src/arrow/util/formatting_util_test.cc Outdated Show resolved Hide resolved

github-actions bot added awaiting changes Awaiting changes and removed awaiting merge Awaiting merge labels Dec 19, 2023

Update test with different resolutions

32f13e8

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 20, 2023

jorisvandenbossche reviewed Dec 20, 2023

View reviewed changes

cpp/src/arrow/util/formatting_util_test.cc Outdated Show resolved Hide resolved

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Dec 20, 2023

Optimise the tests

f53df2c

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 20, 2023

AlenkaF mentioned this pull request Dec 20, 2023

[Python] Add timezone information when printing TimestampArray #39315

Open

AlenkaF requested a review from jorisvandenbossche December 21, 2023 12:40

jorisvandenbossche approved these changes Jan 8, 2024

View reviewed changes

jorisvandenbossche merged commit a288364 into apache:main Jan 8, 2024
33 of 35 checks passed

jorisvandenbossche removed the awaiting change review Awaiting change review label Jan 8, 2024

github-actions bot added the awaiting merge Awaiting merge label Jan 8, 2024

AlenkaF deleted the gh-30117-print-tz branch January 8, 2024 13:32

felipecrv mentioned this pull request Apr 5, 2024

GH-41044: [C++] formatting.h: Make sure space is allocated for the 'Z' when formatting timestamps #41045

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-30117: [C++][Python] Add "Z" to the end of timestamp print string when tz defined #39272

GH-30117: [C++][Python] Add "Z" to the end of timestamp print string when tz defined #39272

AlenkaF commented Dec 18, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Dec 18, 2023

rok left a comment

rok commented Dec 18, 2023

AlenkaF commented Dec 18, 2023

rok commented Dec 18, 2023

jorisvandenbossche commented Dec 19, 2023

rok commented Dec 19, 2023

AlenkaF commented Dec 19, 2023

AlenkaF commented Dec 20, 2023

jorisvandenbossche commented Jan 8, 2024

conbench-apache-arrow bot commented Jan 8, 2024

GH-30117: [C++][Python] Add "Z" to the end of timestamp print string when tz defined #39272

GH-30117: [C++][Python] Add "Z" to the end of timestamp print string when tz defined #39272

Conversation

AlenkaF commented Dec 18, 2023 • edited by github-actions bot Loading

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

github-actions bot commented Dec 18, 2023

rok left a comment

Choose a reason for hiding this comment

rok commented Dec 18, 2023

AlenkaF commented Dec 18, 2023

rok commented Dec 18, 2023

jorisvandenbossche commented Dec 19, 2023

rok commented Dec 19, 2023

AlenkaF commented Dec 19, 2023

AlenkaF commented Dec 20, 2023

jorisvandenbossche commented Jan 8, 2024

conbench-apache-arrow bot commented Jan 8, 2024

AlenkaF commented Dec 18, 2023 •

edited by github-actions bot

Loading