Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Asc/desc hotfix for range partitioning #1829

Closed
jaychia opened this issue Feb 1, 2024 · 0 comments · Fixed by #1862
Closed

[BUG] Asc/desc hotfix for range partitioning #1829

jaychia opened this issue Feb 1, 2024 · 0 comments · Fixed by #1862
Assignees

Comments

@jaychia
Copy link
Contributor

jaychia commented Feb 1, 2024

No description provided.

@jaychia jaychia added this to Daft-OSS Feb 1, 2024
@jaychia jaychia converted this from a draft issue Feb 1, 2024
@clarkzinzow clarkzinzow moved this from In progress to In Review in Daft-OSS Feb 9, 2024
clarkzinzow added a commit that referenced this issue Feb 14, 2024
…for range partitioning and sorting. (#1862)

This PR ensures that we properly track ascending/descending sort order
for range partitioning and sorting ops, to ensure that downstream
operations correctly interpret the partition spec.

For example, before this PR, the planner would always assume that if
both sides of the join are sorted, then they're sorted in ascending
order and could therefore be efficiently joined with our sort-merge join
(which currently only does ascending-order sorts/merges); this would
lead to incorrect results if either side was sorted in descending order.

Closes #1829 

## Follow-ups (future PRs)

- Refactor sort-merge join to work for both ascending and descending
order; this can be done when porting the `Partitioning` abstraction to
Rust to enable better dtype support.
@github-project-automation github-project-automation bot moved this from In Review to Done in Daft-OSS Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Archived in project
2 participants