Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(dask): avoid unnecessary calls to compute for cov/corr/argmin/argmax #9015

Merged
merged 2 commits into from
Apr 18, 2024

Conversation

jcrist
Copy link
Member

@jcrist jcrist commented Apr 18, 2024

Avoids calling compute when executing cov/corr/argmin/argmax.

This necessarily drops support for cov/corr with how="pop", since dask doesn't include native support (yet).

Fixes #8187.

jcrist added 2 commits April 18, 2024 14:54
BREAKING CHANGE: the dask backend no longer supports `cov`/`corr` with `how="pop"`.
@cpcloud cpcloud added this to the 9.0 milestone Apr 18, 2024
@cpcloud cpcloud added performance Issues related to ibis's performance dask The Dask backend labels Apr 18, 2024
@jcrist jcrist enabled auto-merge (rebase) April 18, 2024 20:40
@jcrist jcrist merged commit 1204c56 into ibis-project:main Apr 18, 2024
84 of 86 checks passed
@jcrist jcrist deleted the dask-fixups branch April 18, 2024 21:02
@cpcloud
Copy link
Member

cpcloud commented Apr 18, 2024

Hm, looks like some failures made it through 🤔

@jcrist
Copy link
Member Author

jcrist commented Apr 18, 2024

Weird. I clicked "enable auto-merge", wouldn't have expected that to merge if CI wasn't completely green.

@cpcloud
Copy link
Member

cpcloud commented Apr 18, 2024

Yep, looking into it.

@cpcloud
Copy link
Member

cpcloud commented Apr 18, 2024

For some reason the backends "summary" job was skipped. I think it might be because our ibis-backends-skip-helper.yml workflow's job names don't exactly match our ibis-backends.yml workflow's job names.

This is a hairy part of GHA that is both hard to reason about and test 😬

cpcloud added a commit that referenced this pull request Apr 19, 2024
I _think_ this should prevent weird merge-even-though-the-things-failed
problems observed in #9015.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dask The Dask backend performance Issues related to ibis's performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

perf(dask): avoid calling compute for ops.Covariance, ops.ArgMin and ops.ArgMax
2 participants