Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support pagination in V2 engine, phase 1 #226

Merged
merged 47 commits into from
Mar 28, 2023
Merged

Support pagination in V2 engine, phase 1 #226

merged 47 commits into from
Mar 28, 2023

Commits on Mar 3, 2023

  1. Fixing integration tests broken during POC 

    Signed-off-by: MaxKsyunz <[email protected]>
    @Yury-Fridlyand
    MaxKsyunz authored and Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    7e2dc80 View commit details
    Browse the repository at this point in the history

  2. Comment to clarify an exception. 

    Signed-off-by: MaxKsyunz <[email protected]>
    @Yury-Fridlyand
    MaxKsyunz authored and Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    d359751 View commit details
    Browse the repository at this point in the history

  3. Add support for paginated scroll request, first page. 

    Implement PaginatedPlanCache.convertToPlan for second page to work.

Signed-off-by: MaxKsyunz <[email protected]>
    @Yury-Fridlyand
    MaxKsyunz authored and Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    9c3f7fe View commit details
    Browse the repository at this point in the history

  4. Progress on paginated scroll request, subsequent page. 

    Signed-off-by: MaxKsyunz <[email protected]>
    @Yury-Fridlyand
    MaxKsyunz authored and Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    1ee718b View commit details
    Browse the repository at this point in the history

  5. Move ExpressionSerializer from opensearch to core. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    f3cade6 View commit details
    Browse the repository at this point in the history

  6. Rename Cursor asString to toString. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    db342b8 View commit details
    Browse the repository at this point in the history

  7. Disable scroll cleaning. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    c8f0935 View commit details
    Browse the repository at this point in the history

  8. Add full cursor serialization and deserialization. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    fffc36d View commit details
    Browse the repository at this point in the history

  9. Misc fixes. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    d844977 View commit details
    Browse the repository at this point in the history

  10. Further work on pagination. 

    * Added push down page size from `LogicalPaginate` to `LogicalRelation`.
* Improved cursor encoding and decoding.
* Added cursor compression.
* Fixed issuing `SearchScrollRequest`.
* Fixed returning last empty page.
* Minor code grooming/commenting.

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    333432e View commit details
    Browse the repository at this point in the history

  11. Pagination fix for empty indices. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    85c8825 View commit details
    Browse the repository at this point in the history

  12. Fix error reporting on wrong cursor. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    484a8fe View commit details
    Browse the repository at this point in the history

  13. Minor comments and error reporting improvement. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    cccce53 View commit details
    Browse the repository at this point in the history

  14. Add an end-to-end integration test. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    2895883 View commit details
    Browse the repository at this point in the history

  15. Add explain request handlers. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    dd6fcd6 View commit details
    Browse the repository at this point in the history

  16. Add IT for explain. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    2a19e56 View commit details
    Browse the repository at this point in the history

  17. Address issues flagged by checkstyle build step (#229) 

    Signed-off-by: MaxKsyunz <[email protected]>
    @Yury-Fridlyand
    Max Ksyunz authored and Yury-Fridlyand committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    a3ef2bf View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2023

  1. Pagination, phase 1: Add unit tests for :core module with coverage. (… 

    …#230)

* Add unit tests for `:core` module with coverage. Uncovered: `toCursor`, because it is will be changed soon.

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand authored Mar 8, 2023
    Configuration menu
    Copy the full SHA
    2d29549 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2023

  1. Pagination, phase 1: Add unit tests for SQL module with coverage. (#239) 

    * Add unit tests for SQL module with coverage.

Signed-off-by: Yury-Fridlyand <[email protected]>

* Update sql/src/main/java/org/opensearch/sql/sql/domain/SQLQueryRequest.java

Signed-off-by: Yury-Fridlyand <[email protected]>

Co-authored-by: GabeFernandez310 <[email protected]>

---------

Signed-off-by: Yury-Fridlyand <[email protected]>
Co-authored-by: GabeFernandez310 <[email protected]>
    @Yury-Fridlyand @GabeFernandez310
    Yury-Fridlyand and GabeFernandez310 authored Mar 10, 2023
    Configuration menu
    Copy the full SHA
    70ccfcb View commit details
    Browse the repository at this point in the history

  2. Pagination, phase 1: Add unit tests for :opensearch module with cov… 

    …erage. (#233)

* Add UT for `:opensearch` module with full coverage, except `toCursor`.

Signed-off-by: Yury-Fridlyand <[email protected]>

* Fix checkstyle.

Signed-off-by: Yury-Fridlyand <[email protected]>

---------

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand authored Mar 10, 2023
    Configuration menu
    Copy the full SHA
    803f50e View commit details
    Browse the repository at this point in the history

  3. Fix the merges. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 10, 2023
    Configuration menu
    Copy the full SHA
    27e1793 View commit details
    Browse the repository at this point in the history

  4. Fix explain. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 10, 2023
    Configuration menu
    Copy the full SHA
    304616d View commit details
    Browse the repository at this point in the history

  5. Fix scroll cleaning. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 10, 2023
    Configuration menu
    Copy the full SHA
    1b5ab7e View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2023

  1. Store TotalHits and use it to report total in response. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    f4ea4ad View commit details
    Browse the repository at this point in the history

  2. Add missing UT for :protocol module. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    7f0acdd View commit details
    Browse the repository at this point in the history

  3. Fix PPL UTs damaged in f4ea4ad. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    2ce1626 View commit details
    Browse the repository at this point in the history

  4. Minor checkstyle fixes. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 11, 2023
    Configuration menu
    Copy the full SHA
    b2e6e56 View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2023

  1. Fallback to v1 engine for pagination (#245) 

    * Pagination fallback integration tests.


Signed-off-by: MaxKsyunz <[email protected]>
    @MaxKsyunz
    MaxKsyunz authored Mar 13, 2023
    Configuration menu
    Copy the full SHA
    c7ad219 View commit details
    Browse the repository at this point in the history

  2. Add UT with coverage for toCursor serialization. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 13, 2023
    Configuration menu
    Copy the full SHA
    981bc25 View commit details
    Browse the repository at this point in the history

  3. Fix broken tests in legacy. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 13, 2023
    Configuration menu
    Copy the full SHA
    960c039 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2023

  1. Fix getting total from non-paged requests and from queries without … 

    …`FROM` clause.

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    4f0c176 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2023

  1. Fix scroll cleaning. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    bdd52a0 View commit details
    Browse the repository at this point in the history

  2. Fix cursor request processing. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    a16332f View commit details
    Browse the repository at this point in the history

  3. Update ITs. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    9f9e873 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2023

  1. Fix (again) TotalHits feature. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    3340e38 View commit details
    Browse the repository at this point in the history

  2. Fix typo in prometheus config. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    524f220 View commit details
    Browse the repository at this point in the history

  3. Recover commented logging. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    281f3cd View commit details
    Browse the repository at this point in the history

  4. Move test_pagination_blackbox to a separate class and add logging. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    ca76e1b View commit details
    Browse the repository at this point in the history

  5. Address some PR feedbacks: rename some classes and revert unnecessary… 

    … whitespace changed.

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    c8fcd4e View commit details
    Browse the repository at this point in the history

  6. Minor commenting. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    ec5fb40 View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2023

  1. Address PR comments. 

    * Add javadocs
* Renames
* Cleaning up some comments
* Remove unused code
* Speed up IT

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    4213388 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2023

  1. Minor missing changes. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    ca20d16 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2023

  1. Integration tests for fetch_size, max_result_window, and query.size_l… 

    …imit (#248)


Signed-off-by: MaxKsyunz <[email protected]>
    Max Ksyunz authored Mar 27, 2023
    Configuration menu
    Copy the full SHA
    a58733e View commit details
    Browse the repository at this point in the history

  2. Remove PaginatedQueryService, extend QueryService to hold two pla… 

    …nners and use them.

Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    75b8140 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2023

  1. Move push down functions from request builders to a new interface. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    ba81407 View commit details
    Browse the repository at this point in the history

  2. Some file moves. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    cec012b View commit details
    Browse the repository at this point in the history

  3. Minor clean-up according to PR review. 

    Signed-off-by: Yury-Fridlyand <[email protected]>
    @Yury-Fridlyand
    Yury-Fridlyand committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    9032b3b View commit details
    Browse the repository at this point in the history