Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Faster encoding functions. #8565

Merged
merged 4 commits into from
Jan 4, 2024
Merged

Faster encoding functions. #8565

merged 4 commits into from
Jan 4, 2024

Conversation

dcherian
Copy link
Contributor

@dcherian dcherian commented Dec 21, 2023 •
edited
Loading

Spotted when profiling some write workloads.

  1. Speeds up the check for multi-index
  2. Speeds up one string encoder by not re-creating variables when not necessary.

@benbovy is there a better way?

@benbovy
Copy link
Member

benbovy commented Dec 21, 2023

isinstance(var._data, xarray.core.indexing.PandasMultiIndexingAdapter) should work.

@dcherian dcherian marked this pull request as ready for review December 21, 2023 18:06
@dcherian dcherian changed the title Faster ensure_not_multiindex Faster encoding functions. Dec 21, 2023
@dcherian dcherian requested a review from kmuehlbauer January 3, 2024 17:28
kmuehlbauer
kmuehlbauer approved these changes Jan 4, 2024
View reviewed changes
Copy link
Contributor

@kmuehlbauer kmuehlbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coding part LGTM, 👍

@dcherian dcherian merged commit 5f1f78f into pydata:main Jan 4, 2024
27 checks passed
@dcherian dcherian deleted the optimize-encode-multiindex-check branch January 4, 2024 14:25
dcherian added a commit to dcherian/xarray that referenced this pull request Jan 4, 2024
@dcherian
Merge remote-tracking branch 'upstream/main' into init-zarr
1498c35 
* upstream/main:
  Faster encoding functions. (pydata#8565)
  ENH: vendor SerializableLock from dask and use as default backend lock, adapt tests (pydata#8571)
  Silence a bunch of CachingFileManager warnings (pydata#8584)
  Bump actions/download-artifact from 3 to 4 (pydata#8556)
  Minimize duplication in `map_blocks` task graph (pydata#8412)
  [pre-commit.ci] pre-commit autoupdate (pydata#8578)
  ignore a `DeprecationWarning` emitted by `seaborn` (pydata#8576)
  Fix mypy type ignore (pydata#8564)
  Support for the new compression arguments. (pydata#7551)
  FIX: reverse index output of bottleneck move_argmax/move_argmin functions (pydata#8552)
@benbovy benbovy mentioned this pull request Jan 26, 2024
Closed
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kmuehlbauer kmuehlbauer kmuehlbauer approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dcherian @benbovy @kmuehlbauer