Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rptest: perform full log reads with KgoVerifierSeqConsumer #11759

Merged
merged 5 commits into from
Jul 6, 2023

Commits on Jul 4, 2023

  1. rptest: make kgo seq consumer read full log

    Previously, whenever a KgoVerifierSequentialConsumer and a
    KgoVerifierProducer ran in parallel (which a number of tests do), there
    was no guarantee that the consumer would read all partitions entirely.
    Whenever `wait` is called on the consumer, the `last_pass` command is
    triggered without taking into account if the producer has completed.
    
    This is problematic because it can hide stuck consumer issues. This
    patch makes use of the enhanced kgo-verifier status which now contains
    the high watermark for each partition. When creating a sequential
    consumer, a reference to the producer can be provided. If that was the
    case, when `wait` is called on the consumer we will wait until the
    producer has finished and the high watermarks match.
    Vlad Lazar committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    a33520b View commit details
    Browse the repository at this point in the history
  2. rptest: consume full log in timing stress tests

    Vlad Lazar committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    a90ab2e View commit details
    Browse the repository at this point in the history
  3. rptest: upload manifests more eagerly in timing test

    This commit updates the SI configuration usesd for the
    CloudStorageTimingStressTest(s) to upload the manifest at each
    oportunity. The change should make the test more robust.
    Vlad Lazar committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    2cce635 View commit details
    Browse the repository at this point in the history
  4. dt/docker: bump kgo-verifier version for HWMs

    Vlad Lazar committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    2893f9a View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. Configuration menu
    Copy the full SHA
    fa49a04 View commit details
    Browse the repository at this point in the history