add TakeOrderedOperator #2863
add TakeOrderedOperator #2863
Conversation
qianheng-aws
requested review from
ps48,
kavithacm,
derek-ho,
joshuali925,
dai-chen,
YANG-DB,
rupal-bq,
mengweieric,
vamsimanohar,
Swiddis,
penghuo,
seankao-az,
MaxKsyunz,
Yury-Fridlyand,
anirudha,
forestmvey,
acarbonetto,
GumpacG,
ykmr1224 and
LantaoJin
as code owners
Signed-off-by: Heng Qian <[email protected]>
Signed-off-by: Heng Qian <[email protected]>
qianheng-aws
force-pushed
the
takeOrderedOperator
branch
from
6292fe6 to
b03cfc0
Compare
Signed-off-by: Heng Qian <[email protected]>
core/src/main/java/org/opensearch/sql/planner/physical/TakeOrderedOperator.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/planner/physical/TakeOrderedOperator.java
Show resolved
Hide resolved
Signed-off-by: Heng Qian <[email protected]>
Signed-off-by: Heng Qian <[email protected]>
Signed-off-by: Heng Qian <[email protected]>
qianheng-aws
force-pushed
the
takeOrderedOperator
branch
from
abf1b9c to
4b25b72
Compare
| this.sortList = sortList; | ||
| this.limit = limit; | ||
| this.offset = offset; | ||
| this.ordering = SortHelper.constructExprOrdering(sortList); |
There was a problem hiding this comment.
good job on simpification!
Signed-off-by: Heng Qian <[email protected]>
…redOperator # Conflicts: # docs/user/optimization/optimization.rst
Signed-off-by: Heng Qian <[email protected]>
| assertThat( | ||
| execute( | ||
| takeOrdered( | ||
| inputPlan, 2, 1, Pair.of(SortOption.DEFAULT_ASC, ref("response", INTEGER)))), |
There was a problem hiding this comment.
Could you add a test which its limit equals its offset?
There was a problem hiding this comment.
Oh, it comes to me that we may not need to do that test since Limit(offset, limit) in OpenSearch has different semantics with other engine. For OpenSearch, limit stands for the size of HEAD and offset stands for the from of HEAD.
So it will always return limit rows no matter which value offset is.
e.g.
POST _plugins/_ppl
{
"query": """
source=opensearch_dashboards_sample_data_flights| fields FlightTimeMin | head 2 from 2
"""
}
# response
{
"schema": [
{
"name": "FlightTimeMin",
"type": "float"
}
],
"datarows": [
[
0
],
[
222.74905
]
],
"total": 2,
"size": 2
}
There was a problem hiding this comment.
So it will always return limit rows no matter which value offset is.
Will skip(offset) in open() method of TakeOrderedOperator change the current behavior?
Nice to have a test to compare the results between take-ordered and limit+head.
There was a problem hiding this comment.
Sorry, it's not exact right to say it will always return limit rows no matter which value offset is actually. It's right only when the size of input rows is no less than offset + limit.
I think skip(offset) is semantic equal to what LimitOperator do in its open(). You can see the expected results for each UT, I've add different limit, offset cases for each sort options. But I'll add one more test to verify that's also what sort + limit will produce.
…limit Signed-off-by: Heng Qian <[email protected]>
Signed-off-by: Heng Qian <[email protected]>
There was a problem hiding this comment.
Thanks for the changes!
In future, I think it'd great to have micro-benchmark in benchmarks module to demonstrate such improvements.
--------- Signed-off-by: Heng Qian <[email protected]> (cherry picked from commit 0e70a50) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* add TakeOrderedOperator (#2863) --------- Signed-off-by: Heng Qian <[email protected]> (cherry picked from commit 0e70a50) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Make it compatible with java11 Signed-off-by: Heng Qian <[email protected]> --------- Signed-off-by: Heng Qian <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
--------- Signed-off-by: Heng Qian <[email protected]>
--------- Signed-off-by: Heng Qian <[email protected]>
Description
add TakeOrderedOperator
Issues Resolved
#2857
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.