Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize "ORDER BY + LIMIT" queries for speed / memory with special TopK operator #7721

Merged
merged 36 commits into from
Oct 5, 2023
Merged

Optimize "ORDER BY + LIMIT" queries for speed / memory with special TopK operator #7721

merged 36 commits into from
Oct 5, 2023

Commits on Aug 11, 2023

  1. Prototype TopK operator

    @alamb
    alamb committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    524af05 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Avoid use of Row

    @alamb
    alamb committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    d4c09f2 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Merge remote-tracking branch 'apache/main' into alamb/topk

    @alamb
    alamb committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    6c85247 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2023

  1. start working on compaction

    @alamb
    alamb committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    948c1a2 View commit details
    Browse the repository at this point in the history

  2. checkpoint

    @alamb
    alamb committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    354d687 View commit details
    Browse the repository at this point in the history

  3. update

    @alamb
    alamb committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    afea7d3 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2023

  1. checkpoint

    @alamb
    alamb committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    69b86ab View commit details
    Browse the repository at this point in the history

  2. fmt

    @alamb
    alamb committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    c8b415c View commit details
    Browse the repository at this point in the history

  3. Fix compaction

    @alamb
    alamb committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    0337e31 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2023

  1. add location for re-encoding

    @alamb
    alamb committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    db196fb View commit details
    Browse the repository at this point in the history

  2. Start sketching dictionary interleave

    @alamb
    alamb committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    f123075 View commit details
    Browse the repository at this point in the history

  3. checkpoint

    @alamb
    alamb committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    157379a View commit details
    Browse the repository at this point in the history

  4. initial specialized dictionary

    @alamb
    alamb committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    682127a View commit details
    Browse the repository at this point in the history

  5. finish initial special interleave

    @alamb
    alamb committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    a1ea62e View commit details
    Browse the repository at this point in the history

  6. Complete dictionary order

    @alamb
    alamb committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    5e65130 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. Merge

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    7f29366 View commit details
    Browse the repository at this point in the history

  2. Merge

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    4a30c4c View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into topk

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    78163bd View commit details
    Browse the repository at this point in the history

  4. fmt

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    d9c596f View commit details
    Browse the repository at this point in the history

  5. Cleanup

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    c0f89c1 View commit details
    Browse the repository at this point in the history

  6. Fix test

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    466d4b6 View commit details
    Browse the repository at this point in the history

  7. Cleanup

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    33065ad View commit details
    Browse the repository at this point in the history

  8. Make test deterministic

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    e31718e View commit details
    Browse the repository at this point in the history

  9. Clippy, doctest

    @Dandandan
    Dandandan committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    40ef448 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Use into_sorted_vec

    @Dandandan
    Dandandan committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    c373ce3 View commit details
    Browse the repository at this point in the history

  2. Fix nondeterministic tests

    @Dandandan
    Dandandan committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    bd72ad8 View commit details
    Browse the repository at this point in the history

  3. Update cargo.lock

    @Dandandan
    Dandandan committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    84ffae8 View commit details
    Browse the repository at this point in the history

  4. Merge

    @Dandandan
    Dandandan committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    21ea10f View commit details
    Browse the repository at this point in the history

  5. Update datafusion/physical-plan/src/topk/mod.rs 

    Co-authored-by: Andrew Lamb <[email protected]>
    @Dandandan @alamb
    Dandandan and alamb authored Oct 3, 2023
    Configuration menu
    Copy the full SHA
    592b10e View commit details
    Browse the repository at this point in the history

  6. Update datafusion/physical-plan/src/topk/mod.rs 

    Co-authored-by: Andrew Lamb <[email protected]>
    @Dandandan @alamb
    Dandandan and alamb authored Oct 3, 2023
    Configuration menu
    Copy the full SHA
    47ee199 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Update datafusion/physical-plan/src/topk/mod.rs 

    Co-authored-by: Andrew Lamb <[email protected]>
    @Dandandan @alamb
    Dandandan and alamb authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    2c33637 View commit details
    Browse the repository at this point in the history

  2. Update datafusion/physical-plan/src/topk/mod.rs 

    Co-authored-by: Andrew Lamb <[email protected]>
    @Dandandan @alamb
    Dandandan and alamb authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    c9121cc View commit details
    Browse the repository at this point in the history

  3. Add / update some comments

    @Dandandan
    Dandandan committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    0dc3488 View commit details
    Browse the repository at this point in the history

  4. Rename test file

    @Dandandan
    Dandandan committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    0470306 View commit details
    Browse the repository at this point in the history

  5. Rename table as well

    @Dandandan
    Dandandan committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    0c59fe1 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. Update datafusion/sqllogictest/test_files/topk.slt 

    Co-authored-by: Andrew Lamb <[email protected]>
    @Dandandan @alamb
    Dandandan and alamb authored Oct 5, 2023
    Configuration menu
    Copy the full SHA
    6bb299b View commit details
    Browse the repository at this point in the history