You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed some potentially unexpected behaviour when converting from string to date. Days that are out of bounds for the given month are rolled over into the following month.
I think the expected behaviour would be to either error (Python) or return NULL/NA (R), but not to roll over dates in the following month.
library(arrow, warn.conflicts=FALSE)
library(lubridate, warn.conflicts=FALSE)
library(dplyr, warn.conflicts=FALSE)
df<-tibble::tibble(string_date="1999-02-30")
# base R returns NAdf %>%
mutate(date= strptime(string_date, format="%Y-%m-%d"))
#> # A tibble: 1 × 2#> string_date date #> <chr> <dttm>#> 1 1999-02-30 NA# arrow rolls over the 30th of February into the 2nd of Marchdf %>%
arrow_table() %>%
mutate(date= strptime(string_date, format="%Y-%m-%d")) %>%
collect()
#> # A tibble: 1 × 2#> string_date date #> <chr> <dttm> #> 1 1999-02-30 1999-03-02 00:00:00
Thanks Alenka, Joris and Rok for helping me with the Python examples:
pandas:
Joris Van den Bossche / @jorisvandenbossche:
Copying my comment from the other issue: personally, I don't like that behaviour, but I suppose we get this from the system strptime? (so that might even depend on your OS?)
It might be interesting to check what date.h's version of strptime does.
I think the expected behaviour would be to either error (Python) or return NULL/NA (R),
I would say it is both: by default error, and optionally return null (after ARROW-15665)
I noticed some potentially unexpected behaviour when converting from string to date. Days that are out of bounds for the given month are rolled over into the following month.
I think the expected behaviour would be to either error (Python) or return NULL/NA (R), but not to roll over dates in the following month.
Thanks Alenka, Joris and Rok for helping me with the Python examples:
pandas:
datetime:
arrow:
Reporter: Dragoș Moldovan-Grünfeld / @dragosmg
Watchers: Rok Mihevc / @rok
Related issues:
Note: This issue was originally created as ARROW-15948. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: