Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set date_format to ISO8601 to silence pandas 2.0 UserWarning on read_csv #2569

Merged
merged 4 commits into from
Jun 9, 2023

Conversation

weiji14
Copy link
Member

@weiji14 weiji14 commented Jun 9, 2023

Description of proposed changes

Use pd.read_csv(..., date_format="ISO8601") to silence UserWarning: Could not infer format, so each element will be parsed individually, falling back to dateutil. To ensure parsing is consistent and as-expected, please specify a format..

Note that this requires pandas>=2.0 to work.

References:

Fixes #2480

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
  • If adding new functionality, add an example to docstrings or tutorials.
  • Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash commands are:

  • /format: automatically format and lint the code
  • /test-gmt-dev: run full tests on the latest GMT development version

Use `pd.read_csv(..., date_format="ISO8601")` to silence `UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.`. Xref pandas-dev/pandas#50601
@weiji14 weiji14 added the maintenance Boring but important stuff for the core devs label Jun 9, 2023
@weiji14 weiji14 added this to the 0.10.0 milestone Jun 9, 2023
@weiji14 weiji14 self-assigned this Jun 9, 2023
@@ -235,6 +235,7 @@ def x2sys_cross(tracks=None, outfile=None, **kwargs):
header=2, # Column names are on 2nd row
comment=">", # Skip the 3rd row with a ">"
parse_dates=[2, 3], # Datetimes on 3rd and 4th column
date_format="ISO8601",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This date_format parameter only works with pandas>=2.0. Do we want to set a minimum version of pandas, or make things backwards compatible with pandas=1.x?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make things backwards compatible with pandas=1.x?

This would be better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, compatibility with pandas=1.x and 2.x should be done in fa2a6d3 and 146a7f4. I've also changed ci_tests_legacy to run with pandas<2 as a check.

weiji14 added 2 commits June 9, 2023 16:32
Only use the date_format="ISO8601" argument when pandas>=2.0.0 is used. Set CI Tests Legacy to run with pandas<2 to ensure this works.
@weiji14 weiji14 marked this pull request as ready for review June 9, 2023 05:17
@seisman seisman merged commit de82ef6 into main Jun 9, 2023
@seisman seisman deleted the pandas-2.0/date_format branch June 9, 2023 05:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pandas 2.0.0 brings some warnings and incompatible changes
2 participants