Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Incorrect parsing of ISO 8601 durations #36204

Closed
1 task
lmeyerov opened this issue Sep 7, 2020 · 3 comments · Fixed by #37159
Closed
1 task

BUG: Incorrect parsing of ISO 8601 durations #36204

lmeyerov opened this issue Sep 7, 2020 · 3 comments · Fixed by #37159
Labels
Bug Timedelta Timedelta data type
Milestone

Comments

@lmeyerov
Copy link

lmeyerov commented Sep 7, 2020

  • [x ] I have checked that this issue has not already been reported.

  • [ x] I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

pd.Timedelta('P1Y')
pd.Timedelta('P1M')
pd.Timedelta('P1W')
pd.Timedelta('P1D')
pd.Timedelta('P1DT1H')
pd.Timedelta('PT1H')
pd.Timedelta('P1H')

Problem description

Trying most iso8601 duration formats fails.

For above examples:

  • Expected pd.Timedelta instances of 1yr, 1mo, 1wk, 1d, 1d 1hr
  • ... but got ValueError: Invalid ISO 8601 Duration format - ... on Y/M/W, PT1H
  • ... 1D was '0'
  • ... 1DT1H was '1d' (no hr)

Encountered while trying to figure out neo4j.time.Duration().isoformat() -> pandas ->arrow > rapids cudf timedelta[*] . Neo4j returns values like P1Y3M15DT2H3M and P1Y3M.

@lmeyerov lmeyerov added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 7, 2020
@jreback
Copy link
Contributor

jreback commented Sep 7, 2020

Y M are not valid Timedeltas (as they are not fixed intervals)

they the other might be a bug if they are in fact ISO 8601

pull requests welcome

@jreback jreback added Timedelta Timedelta data type and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 7, 2020
@jreback jreback added this to the Contributions Welcome milestone Sep 7, 2020
@lmeyerov
Copy link
Author

lmeyerov commented Sep 7, 2020

This sounds like pandas docs should not say they support ISO durations

Pandas date time guide:

image

vs wikipedia summary of iso8601 durations:

image

@lmeyerov
Copy link
Author

lmeyerov commented Sep 7, 2020

Linking:

#29773

#15136

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timedelta Timedelta data type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants