Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(page_service): timeout-based batching of requests #9321

Merged
merged 32 commits into from
Nov 18, 2024
Merged

feat(page_service): timeout-based batching of requests #9321

merged 32 commits into from
Nov 18, 2024

Commits on Oct 8, 2024

  1. pageserver: merge get page requests 

    Authored by @problame during Lisbon hackathon
    @VladLazar
    VladLazar committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    135cc64 View commit details
    Browse the repository at this point in the history

  2. pageserver: add debounce timeout pageserver config

    @VladLazar
    VladLazar committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e452e14 View commit details
    Browse the repository at this point in the history

  3. pageserver: remove batched requests metric

    @VladLazar
    VladLazar committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ff86639 View commit details
    Browse the repository at this point in the history

  4. pageserver: clean up tracing a bit

    @VladLazar
    VladLazar committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    96afede View commit details
    Browse the repository at this point in the history

  5. ci: cargo fmt

    @VladLazar
    VladLazar committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6aa9a93 View commit details
    Browse the repository at this point in the history

  6. ci: clippy

    @VladLazar
    VladLazar committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    f8e222a View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. pageserver/tests: instrument get rel page calls

    @VladLazar
    VladLazar committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    31530ab View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. review: update naming to use batching

    @VladLazar
    VladLazar committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5b36b76 View commit details
    Browse the repository at this point in the history

  2. review: refactor batching into separate function 

    The refactor is straightforward, but we also fix a bug.
Previously, a batch could have been empty if handling the first message
resulted in an error (would have panicked). This has been fixed by
including errors in the current batch.
    @VladLazar
    VladLazar committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    fd4a1c8 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. Merge branch 'main' into vlad/pageserver-merge-get-page-requests 

    Conflicts: Pageserver config was also updated in main
    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    2188191 View commit details
    Browse the repository at this point in the history

  2. review: tweak get_rel_page_at_lsn_batched comment

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    66241c4 View commit details
    Browse the repository at this point in the history

  3. review: remove sorting of vectored get keys

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    ade3e77 View commit details
    Browse the repository at this point in the history

  4. review: remove some redundant logging 

    We already log get page errors the pagestream level
and on the ingest path.
    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    0a554e0 View commit details
    Browse the repository at this point in the history

  5. sq: update name to use batching

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    303b372 View commit details
    Browse the repository at this point in the history

  6. review: rewrite response filling logic to avoid extra copy 

    Switch to over to using Vec for the `Vec::spare_capacity_mut`
interface. Fill the `MayebUninit` slots carefully and `Vec::set_len`
at the end.
    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    a983a49 View commit details
    Browse the repository at this point in the history

  7. review: remove stale wip comment

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    b514d68 View commit details
    Browse the repository at this point in the history

  8. ci: fix rust docs gen

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    70aac4b View commit details
    Browse the repository at this point in the history

  9. fixup: return early when batching is disabled

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    46e2095 View commit details
    Browse the repository at this point in the history

  10. ci: add db creation support back

    @VladLazar
    VladLazar committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    9507ebc View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2024

  1. use smallvec inside BatchOrEof

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    39510b2 View commit details
    Browse the repository at this point in the history

  2. https://github.com/neondatabase/neon/pull/9321#discussion_r1812699096

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    431b7a7 View commit details
    Browse the repository at this point in the history

  3. https://github.com/neondatabase/neon/pull/9321#discussion_r1812714554

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    cc97115 View commit details
    Browse the repository at this point in the history

  4. https://github.com/neondatabase/neon/pull/9321#discussion_r1812728994

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    4dba135 View commit details
    Browse the repository at this point in the history

  5. https://github.com/neondatabase/neon/pull/9321#discussion_r1812731796

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    800d412 View commit details
    Browse the repository at this point in the history

  6. https://github.com/neondatabase/neon/pull/9321#discussion_r1812734240

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    7ae217b View commit details
    Browse the repository at this point in the history

  7. https://github.com/neondatabase/neon/pull/9321#discussion_r1812726944

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    e9ab0ff View commit details
    Browse the repository at this point in the history

  8. create all the spans inside read_batch_from_connection; #9321 (comment)

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    8c17488 View commit details
    Browse the repository at this point in the history

  9. remove todo

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    294e189 View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'origin/main' into vlad/pageserver-merge… 

    …-get-page-requests

Conflicts (all syntactic, not semantic)
	libs/pageserver_api/src/config.rs
	pageserver/Cargo.toml
	pageserver/src/config.rs
	pageserver/src/page_service.rs
	pageserver/src/pgdatadir_mapping.rs
    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    2c64378 View commit details
    Browse the repository at this point in the history

  11. python codestyle

    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    c4500c4 View commit details
    Browse the repository at this point in the history

  12. address the CI failures: they were all due to incorrect classificatio… 

    …n of PageReconstructError=>PageStreamError for case of Timeline cancellation
    @problame
    problame committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    234799e View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. use humantime for config flag

    @problame
    problame committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    2bea398 View commit details
    Browse the repository at this point in the history