Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address performance and deprecations in interp_hybrid_to_pressure #592

Merged
merged 8 commits into from
Apr 22, 2024

Conversation

kafitzgerald
Copy link
Contributor

@kafitzgerald kafitzgerald commented Apr 9, 2024

PR Summary

Refactors the helper function _vertical_remap_extrap to avoid problematic for loops over the vertical levels and addresses some warnings related to specification of chunks, dimension specification with argmax, and intentional interpolation out of bounds.

While this seems to address the most significant issues in terms of performance / execution time, there's still some clean up to be done in interp_hybrid_to_pressure and it would be nice to have a benchmark to cover this.

I did also replicate the poor performance with a larger example on my laptop in addition to the HPC system.

Closes #591

PR Checklist

General

  • Make an issue if one doesn't already exist
  • Link the issue this PR resolves by adding closes #XXX to the PR description where XXX is the number of the issue.
  • Add a brief summary of changes to docs/release-notes.rst in a relevant section for the next unreleased release. Possible sections include: Documentation, New Features, Bug Fixes, Internal Changes, Breaking Changes/Deprecated
  • Add appropriate labels to this PR
  • Make your changes in a forked repository rather than directly in this repo
  • Open this PR as a draft if it is not ready for review
  • Convert this PR from a draft to a full PR before requesting reviewers
  • Passes precommit. To set up on your local, run pre-commit install from the top level of the repository. To manually run pre-commits, use pre-commit run --all-files and re-add any changed files before committing again and pushing.
  • If needed, squash and merge PR commits into a single commit to clean up commit history

@kafitzgerald kafitzgerald added bug Something isn't working refactor Internal code refactoring scalability and performance Related to scalability, performance, or benchmarking labels Apr 9, 2024
@kafitzgerald kafitzgerald force-pushed the issue591_interp_dask branch from 87631dd to 677dfc9 Compare April 19, 2024 19:24
@kafitzgerald kafitzgerald changed the title Remove for loops from _vertical_remap_extrap to address slow execution times Address performance and deprecations in interp_hybrid_to_pressure Apr 19, 2024
@kafitzgerald kafitzgerald marked this pull request as ready for review April 19, 2024 22:00
Copy link
Member

@anissa111 anissa111 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this (rough) notebook for some performance comparison if you're interested

@anissa111 anissa111 merged commit 5a1031c into NCAR:main Apr 22, 2024
12 checks passed
@erogluorhan
Copy link
Collaborator

Great work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working refactor Internal code refactoring scalability and performance Related to scalability, performance, or benchmarking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Slow execution times for interp_hybrid_to_pressure with extrapolation
3 participants