[NSE-574]implement columnar limit #575
Conversation
zhouyuan
force-pushed
the
wip_limit
branch
2 times, most recently
from
998ab09 to
8e06431
Compare
Signed-off-by: Yuan Zhou <[email protected]>
Signed-off-by: Yuan Zhou <[email protected]>
Signed-off-by: Yuan Zhou <[email protected]>
Signed-off-by: Yuan Zhou <[email protected]>
| val delta = iter.next() | ||
| rowCount += delta.numRows | ||
| if (rowCount > limit) { | ||
| val newSize = rowCount - limit |
There was a problem hiding this comment.
The newSize calculation logic can be changed like below codes:
if (rowCount < limit) {
val delta = iter.next()
val preRowCount = rowCount
rowCount += delta.numRows
if (rowCount > limit) {
delta.setNumRows(limit - preRowCount)
}
delta
} else {
throw new NoSuchElementException("End of ColumnarBatch iterator")
}
There was a problem hiding this comment.
cool, thanks for the check
| if (rowCount > limit) { | ||
| val newSize = limit - preRowCount | ||
| delta.setNumRows(newSize) | ||
| } |
There was a problem hiding this comment.
hi, delta should be returned
zhouyuan
force-pushed
the
wip_limit
branch
2 times, most recently
from
dabe74e to
8d9d1cb
Compare
|
hi yuan, here should also be changed into ColumnarLocalLimitExec according to my test. |
right, I just added this change |
Signed-off-by: Yuan Zhou <[email protected]>
What changes were proposed in this pull request?
This patch implements columnar Limit
Signed-off-by: Yuan Zhou [email protected]
How was this patch tested?
local unit tests