-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Deprecate combos in favor of lazy single table verbs #22871
Comments
Extensions to lazy operations on multiple tables seems trickier, but having a lazy |
Why do you want |
Also, IIRC |
Well, it doesn't have to be, but I think it would (at least at the moment) improve performance, because |
I've been meaning to write up a Julep and proof-of-concept, but I would like to get rid of |
@simonbyrne I am curious, I thought this would need a new trait Homogenous() which states that the elements of the iterator have the same shape to avoid the ugly lookahead I had in #16708. |
@simonbyrne I started work on a slicing iterator package. So far it just handles mapslices. I'm still not quite there with performance but I'm not too far off. |
The canary in the coalmine is
mapreducedims
, which is a combo of at least three semantic operations: mapping, reducing, and slicing along dimensions. Base seems to be slowly collecting lazy implementations:Base.Generator
,Iterators.filter
, etc. Of the top of my head, we would want to additionally includereduce
andslicedims
as single table verbs. Going all the way would mean that the code currently inmapreducedims
would simply be a specialized method ofcollect
. This would sweep up #3893. Additionally, we might want to make verbs likesum
return a lazyreduce
.The text was updated successfully, but these errors were encountered: