Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1-sided joins (tick/tick only) #1082

Closed
jhellerstein opened this issue Feb 29, 2024 · 2 comments
Closed

1-sided joins (tick/tick only) #1082

jhellerstein opened this issue Feb 29, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@jhellerstein
Copy link
Contributor

For "tick/tick" join, we'd like to minimize memory pressure by persisting only one side (hopefully the small side), and probing with the other. i.e. a traditional (asymmetric) Hash Join. We achieve this by having a stratum boundary on the "build" side of the join.

Note that this doesn't make sense if both sides have "static" duration, because then we'd need to materialize it both sides anyway across ticks to ensure we join old (before this tick) and new (in this tick) tuples across both sides. (i.e. it's a bimorphism).

HOWEVER, a one-sided static/tick join does make sense. It can be done by simply persisting the static side into a persist_keyed, and streaming the tick side to probe it -- this is essentially a morphism, not a bimorphism -- ie. there's no question of joining old and new.

@jhellerstein jhellerstein added the enhancement New feature or request label Feb 29, 2024
@MingweiSamuel
Copy link
Member

#1058

@MingweiSamuel
Copy link
Member

#1373

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants